Stay Informed

This week, read about:

• The Top 5 Open Source News Stories of 2023.
• SSH Shaken, Not Stirred by Terrapin Vulnerability.
• Cost of a Data Breach Report 2023: Insights, Mitigators and Best Practices.

Key Security, Maintenance, and Features Releases

Non-Security Based Updates

Apache Camel 3.22.0

Apache Spark 3.3.4
Notable changes:
[SPARK-43327]: Trigger committer.setupJob before plan execute in FileFormatWriter#write
[SPARK-43393]: Address sequence expression overflow bug
[SPARK-44547]: Ignore fallback storage for cached RDD migration
[SPARK-44581]: Fix the bug that ShutdownHookManager gets wrong UGI from SecurityManager of ApplicationMaster
[SPARK-44725]: Document spark.network.timeoutInterval
[SPARK-44805]: getBytes/getShorts/getInts/etc. should work in a column vector that has a dictionary
[SPARK-44857]: Fix getBaseURI error in Spark Worker LogPage UI buttons
[SPARK-44871]: Fix percentile_disc behaviour
[SPARK-44920]: Use await() instead of awaitUninterruptibly() in TransportClientFactory.createClient()
[SPARK-44925]: K8s default service token file should not be materialized into token
[SPARK-44935]: Fix RELEASE file to have the correct information in Docker images if exists
[SPARK-44937]: Mark connection as timedOut in TransportClient.close
[SPARK-44973]: Fix ArrayIndexOutOfBoundsException in conv()
[SPARK-44990]: Reduce the frequency of get spark.sql.legacy.nullValueWrittenAsQuotedEmptyStringCsv
[SPARK-45057]: Avoid acquire read lock when keepReadLock is false
[SPARK-45079]: Fix an internal error from percentile_approx() on NULL accuracy
[SPARK-45100]: Fix an internal error from reflect()on NULL class and method
[SPARK-45187]: Fix WorkerPage to use the same pattern for logPage urls
[SPARK-45227]: Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
[SPARK-45389]: Correct MetaException matching rule on getting partition metadata
[SPARK-45430]: Fix for FramelessOffsetWindowFunction when IGNORE NULLS and offset > rowCount
[SPARK-45508]: Add “–add-opens=java.base/jdk.internal.ref=ALL-UNNAMED” so Platform can access Cleaner on Java 9+
[SPARK-45580]: Handle case where a nested subquery becomes an existence join
[SPARK-45670]: SparkSubmit does not support --total-executor-cores when deploying on K8s
[SPARK-45749]: Fix Spark History Server to sort Duration column properly
[SPARK-45920]: group by ordinal should be idempotent
[SPARK-46006]: YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop
[SPARK-46012]: EventLogFileReader should not read rolling logs if app status file is missing
[SPARK-46029]: Escape the single quote, _ and % for DS V2 pushdown
[SPARK-46092]: Don’t push down Parquet row group filters that overflow
[SPARK-46095]: Document REST API for Spark Standalone Cluster
[SPARK-46239]: Hide Jetty infhttps://spark.apache.org/releases/spark-release-3-3-4.html
[SPARK-46286]: Document spark.io.compression.zstd.bufferPool.enabled

Dependency Changes:
[SPARK-45885]: Upgrade ORC to 1.7.10

Grafana
10.2.3
Features and enhancements:

• Auth: Improve groups claim setup docs for AzureAD. #79227, @mgyongyosi
• Alerting: Attempt to retry retryable errors. #79175, @gotjosh
• Unified Alerting: Set max_attempts to 1 by default. #79103, @gotjosh
• Auth: Add anonymous users view and stats. #78965, @Jguer

Bug fixes:

• Alerting: Fix deleting rules in a folder with matching UID in another organization. #79011, @papagian
• CloudWatch: Correctly quote metric names with special characters. #78975, @iwysiu
• DeleteDashboard: Redirect to home after deleting a dashboard. #78936, @ivanortegaalba
• Alerting: Fixes combination of multiple predicates for rule search. #78912, @gillesdemey
• CloudWatch: Fetch Dimension keys correctly from Dimension Picker. #78831, @iwysiu
• Tempo: Fix read-only access error. #78801, @fabrizio-grafana
• Bug: Fix broken ui components when angular is disabled. #78670, @jackw
• InfluxDB: Parse data for table view to have parity with frontend parser. #78551, @itsmylife
• Elasticsearch: Fix processing of raw_data with not-recognized time format. #78380, @ivanahuckova
• Recorded Queries: Add org isolation (remote write target per org), and fix cross org Delete/List. (Enterprise)
• Auditing: Fix missing action in alert manager routes. (Enterprise)

10.1.6
Features and enhancements:

• Alerting: Attempt to retry retryable errors. #79211, @gotjosh
• Unified Alerting: Set max_attempts to 1 by default. #79102, @gotjosh

Bug fixes:

• Alerting: Fix deleting rules in a folder with matching UID in another organization. #79007, @papagian
• Chore: Fix timeout issues when gathering prometheus datasource stats. #78858, @DanCech
• Provisioning: Ensure that enterprise provisioning runs [10.1.x]. #76686, @IevaVasiljeva
• Alerting: Make shareable alert rule link work if rule name contains forward slashes. #75950, @domasx2
• Loki: Cache extracted labels. #75905, @gtk-grafana
• DataSourcePicker: Disable autocomplete for the search input . #75900, @ivanortegaalba
• Plugins: Refresh plugin info after installation. #75225, @oshirohugo
• LDAP: FIX Enable users on successfull login . #75176, @gamab
• Loki: Fix filters not being added with multiple expressions and parsers. #75172, @svennergr
• Recorded Queries: Add org isolation (remote write target per org), and fix cross org Delete/List. (Enterprise)
• Auditing and UsageInsights: FIX Loki configuration to use proxy env variables. (Enterprise)

10.0.10
Features and enhancements:

• Alerting: Attempt to retry retryable errors. #79210, @gotjosh
• Unified Alerting: Set max_attempts to 1 by default. #79101, @gotjosh

Bug fixes:

• Recorded Queries: Add org isolation (remote write target per org), and fix cross org Delete/List. (Enterprise)

9.5.15
Features and enhancements:

• Alerting: Attempt to retry retryable errors. #79209, @gotjosh
• Unified Alerting: Set to 1 by default. #79109, @gotjosh

Bug fixes:

• Recorded Queries: Add org isolation (remote write target per org), and fix cross org Delete/List. (Enterprise)

Jenkins 2.437

• Add telemetry for basic Java system properties describing the environment. (pull 8787)
• Restyle widget panes. (pull 8761)
• Rework node monitor configuration. (issue 72371)
• Ensure uptime is independent of system clock. (issue 72157)
• Show monitoring data on agent page. (pull 8725)
• Deprecate all configurable options in **Launch agent by connecting it to the controller** (inbound in JCasC), as these are only useful in conjunction with the deprecated jnlpUrl mode. (pull 8762)
• The jnlpUrl ${JENKINS_URL}/computer/${AGENT_NAME}/jenkinsagent.jnlp argument to the agent JAR has been deprecated. Use url ${JENKINS_URL} and name ${AGENT_NAME} instead, potentially also passing in webSocket, tunnel, and/or work directory options as needed. (pull 8773)
• Display strings consistently in the requested language when running Jenkins in a JVM with a non-english locale. (issue 72449)
• Fix nested job link in mobile view. (issue 72288)
• Do not show option to copy items when there are no items visible. (issue 72443)
• Developer: Allow replacing onclick attributes containing inline JS on l:task with datacallback. (issue 60866)
• Allow users to make side panel sticky. (issue 71578)

Keycloak 23.0.3 & 23.0.2 & 22.0.7

Kubernetes
1.28.5
Feature:

• Kubernetes is now built with Go 1.20.12 (#122216, @xmudrii) [SIG Release and Testing]

Bug or Regression:

• Fix panic if there are more terminating pods than active pods (#122267, @kannon92) [SIG Apps]
• Fix: statle smb mount issue when smb file share is deleted and then unmount (#121851, @andyzhangx) [SIG Storage]
• Fixed a regression since 1.27.0 in scheduler framework when running score plugins. The skippedScorePlugins number might be greater than enabledScorePlugins, so when initializing a slice the cap(len(skippedScorePlugins) - len(enabledScorePlugins)) is negative, which is not allowed. (#121667, @kerthcet) [SIG Scheduling]
• Fixes a kube-apiserver log volume regression bug in default 1.27 configurations (introduced in 1.26, activated by the AggregatedDiscoveryEndpoint feature enablement in 1.27) (#122096, @ritazh) [SIG API Machinery]
• Fixes a regression in kube-scheduler memory use in default 1.28 configurations by moving the SchedulerQueueingHints feature gate back to disabled by default. (#122291, @sanposhiho) [SIG Scheduling]
• Fixes an issue where StatefulSet might not restart a pod after eviction or node failure. (#121389, @aleksandra-malinowska) [SIG Apps and Testing]
• The scheduling queue didn't notice any extenders' failures, it could miss some cluster events, and it could end up Pods rejected by Extenders stuck in unschedulable pod pool in 5min in the worst-case scenario. Now, the scheduling queue notices extenders' failures and requeue Pods rejected by Extenders appropriately. (#122045, @sanposhiho) [SIG Scheduling]

Other (Cleanup or Flake):

• Bump distroless-iptables to 0.2.8 based on Go 1.20.11 (#121976, @cpanato) [SIG Testing]
• Makefile and scripts now respect GOTOOLCHAIN and otherwise ensure ./.go-version is used (#122075, @BenTheElder) [SIG Release and Testing]

1.27.9
Feature:

• Kubernetes is now built with Go 1.20.12 (#122217, @xmudrii) [SIG Release and Testing]

Bug or Regression:

• Fixed a regression since 1.27.0 in scheduler framework when running score plugins. The skippedScorePlugins number might be greater than enabledScorePlugins, so when initializing a slice the cap(len(skippedScorePlugins) - len(enabledScorePlugins)) is negative, which is not allowed. (#121666, @kerthcet) [SIG Scheduling]
• Fixes a kube-apiserver log volume regression bug in default 1.27 configurations (introduced in 1.26, activated by the AggregatedDiscoveryEndpoint feature enablement in 1.27) (#122074, @ritazh) [SIG API Machinery]
• Fixes an issue where StatefulSet might not restart a pod after eviction or node failure. (#121389, @aleksandra-malinowska) [SIG Apps and Testing]
• The scheduling queue didn't notice any extenders' failures, it could miss some cluster events, and it could end up Pods rejected by Extenders stuck in unschedulable pod pool in 5min in the worst-case scenario. Now, the scheduling queue notices extenders' failures and requeue Pods rejected by Extenders appropriately. (#122044, @sanposhiho) [SIG Scheduling]

Other (Cleanup or Flake):

• Bump distroless-iptables to 0.2.8 based on Go 1.20.11 (#121975, @cpanato) [SIG Testing]
• Makefile and scripts now respect GOTOOLCHAIN and otherwise ensure ./.go-version is used (#122076, @BenTheElder) [SIG Release and Testing]

MongoDB 7.2.0

Nodejs 21.5.0
Notable Changes:
[0dd53da722] - (SEMVER-MINOR) deps: add simdjson (Yagiz Nizipli) #50322
[9f54987fbc] - module: merge config with package_json_reader (Yagiz Nizipli) #50322
[45e4f82912] - src: move package resolver to c++ (Yagiz Nizipli) #50322

Deprecations:
[26ed4ad01f] - doc: deprecate hash constructor (Marco Ippolito) #51077
[58ca66a1a7] - doc: deprecate dirent.path (Antoine du Hamel) #51020

Commits:
[1bbdbdfbeb] - benchmark: update iterations in benchmark/perf_hooks (Lei Shi) #50869
[087fb0908e] - benchmark: update iterations in benchmark/crypto/aes-gcm-throughput.js (Lei Shi) #50929
[53b16c71fb] - benchmark: update iteration and size in benchmark/crypto/randomBytes.js (Lei Shi) #50868
[38fd0ca753] - benchmark: add undici websocket benchmark (Chenyu Yang) #50586
[b148c43244] - benchmark: add create-hash benchmark (Joyee Cheung) #51026
[fdd8c18f96] - benchmark: update interations and len in benchmark/util/text-decoder.js (Lei Shi) #50938
[a9972057ac] - benchmark: update iterations of benchmark/util/type-check.js (Lei Shi) #50937
[b80bb1329b] - benchmark: update iterations in benchmark/util/normalize-encoding.js (Lei Shi) #50934
[dbee03d646] - benchmark: update iterations in benchmark/util/inspect-array.js (Lei Shi) #50933
[f2d83a3a84] - benchmark: update iterations in benchmark/util/format.js (Lei Shi) #50932
[2581fce553] - bootstrap: improve snapshot unsupported builtin warnings (Joyee Cheung) #50944
[735bad3694] - build: fix warnings from uv for gn build (Cheng Zhao) #51069
[8da9d969f9] - deps: V8: cherry-pick 0fd478bcdabd (Joyee Cheung) #50572
[429fbb37c1] - deps: update simdjson to v3.6.2 (Yagiz Nizipli) #50986
[9950103253] - deps: update zlib to 1.3-22124f5 (Node.js GitHub Bot) #50910
[0b61823e8b] - deps: update undici to 5.28.2 (Node.js GitHub Bot) #51024
[95d8a273cc] - deps: cherry-pick bfbe4e38d7 from libuv upstream (Abdirahim Musse) #50650
[06038a489e] - deps: update libuv to 1.47.0 (Node.js GitHub Bot) #50650
[0dd53da722] - (SEMVER-MINOR) deps: add simdjson (Yagiz Nizipli) #50322
[04eaa5cdd7] - doc: run license-builder (github-actions[bot]) #51111
[26ed4ad01f] - doc: deprecate hash constructor (Marco Ippolito) #51077
[637ffce4c4] - doc: add note regarding --experimental-detect-module (Shubherthi Mitra) #51089
[838179b096] - doc: correct tracingChannel.traceCallback() (Gerhard Stöbich) #51068
[539bee4f0a] - doc: use length argument in pbkdf2Key (Tobias Nießen) #51066
[c45a9a3187] - doc: add deprecation notice to dirent.path (Antoine du Hamel) #51059
[58ca66a1a7] - doc: deprecate dirent.path (Antoine du Hamel) #51020
[c2b6edf9ab] - esm: fix hook name in error message (Bruce MacNaughton) #50466
[35e8f26f07] - fs: throw fchownSync error from c++ (Yagiz Nizipli) #51075
[c3c8237089] - fs: update params in jsdoc for createReadStream and createWriteStream (Jungku Lee) #51063
[3f7f3ce8c9] - fs: improve error performance of readvSync (IlyasShabi) #50100
[7f95926f17] - http: handle multi-value content-disposition header (Arsalan Ahmad) #50977
[7a8a2d5632] - lib: don't parse windows drive letters as schemes (华) #50580
[aa2be4bb76] - module: load source maps in commonjs translator (Hiroki Osame) #51033
[c0e5e74876] - module: document parentURL in register options (Hiroki Osame) #51039
[4eedf5e694] - module: fix recently introduced coverity warning (Michael Dawson) #50843
[9f54987fbc] - module: merge config with package_json_reader (Yagiz Nizipli) #50322
[5f95dca638] - node-api: introduce experimental feature flags (Gabriel Schulhof) #50991
[3fb7fc909e] - quic: further implementation details (James M Snell) #48244
[fa25e069fc] - src: implement countObjectsWithPrototype (Joyee Cheung) #50572
[abe90527e4] - src: register udp_wrap external references (Joyee Cheung) #50943
[84e2f51d14] - src: register spawn_sync external references (Joyee Cheung) #50943
[2cfee53d7b] - src: register process_wrap external references (Joyee Cheung) #50943
[9b7f79a8bd] - src: fix double free reported by coverity (Michael Dawson) #51046
[fc5503246e] - src: remove unused headers in node_file.cc (Jungku Lee) #50927
[c3abdc58af] - src: implement --trace-promises (Joyee Cheung) #50899
[f90fc83e97] - src: fix dynamically linked zlib version (Richard Lau) #51007
[9bf144379f] - src: omit bool values of package.json main field (Yagiz Nizipli) #50965
[45e4f82912] - src: move package resolver to c++ (Yagiz Nizipli) #50322
[71acd36778] - stream: implement TransformStream cleanup using "transformer.cancel" (Debadree Chatterjee) #50126
[5112306064] - stream: fix fd is null when calling clearBuffer (kylo5aby) #50994
[ed070755ec] - test: deflake test-diagnostics-channel-memory-leak (Joyee Cheung) #50572
[aee01ff1b4] - test: test syncrhnous methods of child_process in snapshot (Joyee Cheung) #50943
[cc949869a3] - test: handle relative https redirect (Richard Lau) #51121
[048349ed4c] - test: fix test runner colored output test (Moshe Atlow) #51064
[7f5291d783] - test: resolve path of embedtest binary correctly (Cheng Zhao) #50276
[4ddd0daf5f] - test: escape cwd in regexp (Jérémy Lal) #50980
[3ccd5faabb] - test_runner: format coverage report for tap reporter (Pulkit Gupta) #51119
[d5c9adf3df] - test_runner: fix infinite loop when files are undefined in test runner (Pulkit Gupta) #51047
[328a41701c] - tools: update lint-md-dependencies to rollup@4.7.0 (Node.js GitHub Bot) #51106
[297cb6f5c2] - tools: update doc to highlight.js@11.9.0 unified@11.0.4 (Node.js GitHub Bot) #50459
[4705023343] - tools: fix simdjson updater (Yagiz Nizipli) #50986
[c9841583db] - tools: update eslint to 8.55.0 (Node.js GitHub Bot) #51025
[2b4671125e] - tools: update lint-md-dependencies to rollup@4.6.1 (Node.js GitHub Bot) #51022
[cd891b37f6] - util: improve performance of function areSimilarFloatArrays (Liu Jia) #51040
[e178a43509] - vm: use v8::DeserializeInternalFieldsCallback explicitly (Joyee Cheung) #50984
[fd028e146f] - win,tools: upgrade Windows signing to smctl (Stefan Stojanovic) #50956

Prometheus 2.45.2
This release contains security fixes in dependencies and has been built with go1.21.5. #13307

• [BUGFIX] TSDB: Fix PostingsForMatchers race with creating new series. #12558

Ceph 18.2.1
NOTABLE CHANGES:

• RGW: S3 multipart uploads using Server-Side Encryption now replicate correctly in a multi-site deployment. Previously, the replicas of such objects were corrupted on decryption. A new command, radosgw-admin bucket resync encrypted multipart, can be used to identify these original multipart uploads. The LastModified timestamp of any identified object is incremented by 1ns to cause peer zones to replicate it again. For multi-site deployments that make any use of Server-Side Encryption, we recommended running this command against every bucket in every zone after all zones have upgraded.
• CEPHFS: MDS now evicts clients which are not advancing their request tids (transaction IDs), which causes a large buildup of session metadata, resulting in the MDS going read-only due to the RADOS operation exceeding the size threshold. mds_session_metadata_threshold config controls the maximum size that an (encoded) session metadata can grow.
• RGW: New tools have been added to radosgw-admin for identifying and correcting issues with versioned bucket indexes. Historical bugs with the versioned bucket index transaction workflow made it possible for the index to accumulate extraneous “book-keeping” olh (object logical head) entries and plain placeholder entries. In some specific scenarios where clients made concurrent requests referencing the same object key, it was likely that a lot of extra index entries would accumulate. When a significant number of these entries are present in a single bucket index shard, they can cause high bucket listing latencies and lifecycle processing failures. To check whether a versioned bucket has unnecessary olh entries, users can now run radosgw-admin bucket check olh. If the --fix flag is used, the extra entries will be safely removed. A distinct issue from the one described thus far, it is also possible that some versioned buckets are maintaining extra unlinked objects that are not listable from the S3/ Swift APIs. These extra objects are typically a result of PUT requests that exited abnormally, in the middle of a bucket index transaction - so the client would not have received a successful response. Bugs in prior releases made these unlinked objects easy to reproduce with any PUT request that was made on a bucket that was actively resharding. Besides the extra space that these hidden, unlinked objects consume, there can be another side effect in certain scenarios, caused by the nature of the failure mode that produced them, where a client of a bucket that was a victim of this bug may find the object associated with the key to be in an inconsistent state. To check whether a versioned bucket has unlinked entries, users can now run radosgw-admin bucket check unlinked. If the --fix flag is used, the unlinked objects will be safely removed. Finally, a third issue made it possible for versioned bucket index stats to be accounted inaccurately. The tooling for recalculating versioned bucket stats also had a bug, and was not previously capable of fixing these inaccuracies. This release resolves those issues and users can now expect that the existing radosgw-admin bucket check command will produce correct results. We recommend that users with versioned buckets, especially those that existed on prior releases, use these new tools to check whether their buckets are affected and to clean them up accordingly.
• mgr/snap-schedule: For clusters with multiple CephFS file systems, all the snap-schedule commands now expect the ‘--fs’ argument.
• RADOS: A POOL_APP_NOT_ENABLED health warning will now be reported if the application is not enabled for the pool irrespective of whether the pool is in use or not. Always tag a pool with an application using ceph osd pool application enable command to avoid reporting of POOL_APP_NOT_ENABLED health warning for that pool. The user might temporarily mute this warning using ceph health mute POOL_APP_NOT_ENABLED.
• Dashboard: An overview page for RGW to show the overall status of RGW components.
• Dashboard: Added management support for RGW Multi-site and CephFS Subvolumes and groups.
• Dashboard: Fixed few bugs and issues around the new dashboard page including the broken layout, some metrics giving wrong values and introduced a popover to display details when there are HEALTH_WARN or HEALTH_ERR.
• Dashboard: Fixed several issues in Ceph dashboard on Rook-backed clusters, and improved the user experience on the Rook environment.