OpenUpdate - September 21, 2023

Stay Informed

This week, read about:

openSUSE: Slowroll.
PostgreSQL 16 Released!
CERN Swaps Out Databases To Feed Its Petabyte-a-Day Habit.
OpenTF Forks Terraform, Insists HashiCorp Is The Splinter Group.
Proactive Software Lifecycle Management for OSS.

Key Security, Maintenance, and Features Releases

Security Based Updates

Security vulnerability when registering or updating user through templates
A security vulnerability was introduced in Keycloak 22.0.2. We highly recommend not upgrading to 22.0.2, and for anyone that has deployed 22.0.2 in production to upgrade to 22.0.3 immediately.
For users that has self-registered after Keycloak was upgraded to 22.0.2 their password is not stored securely, and can be exposed to administrators of Keycloak. This only affects users that has registered after the upgrade was rolled-out, and does not affect any previously registered users.
Any realm using the preview declarative user profile is not affected by this issue, and only realms using the default user profile provider is affected.

Non-Security Based Updates

Artemis 2.31.0
Bug:
[ARTEMIS-4174] - JMX RMI connector-ports limited to localhost listen for remote connections
[ARTEMIS-4370] - Publishing message with existing topic alias and different topic causes message to be sent to incorrect topic
[ARTEMIS-4382] - CLI import / export may take a huge amount of time in large datasets.
[ARTEMIS-4387] - Empty consumer filter string leak
[ARTEMIS-4389] - The word "mesage" should be corrected to "message"
[ARTEMIS-4390] - Windows build fails smoke tests on upgrade-linux
[ARTEMIS-4394] - management console war file contains some duplicate jars
[ARTEMIS-4397] - Problem with bootstrap.xml after artemis upgrade
[ARTEMIS-4399] - Authentication cache set to size 0 (i.e. disabled) is not threadsafe
[ARTEMIS-4400] - artemis-cdi-client: artemis-unit-test-support should be in test scope
[ARTEMIS-4405] - Incorrect username logging in AMQ601264 events
[ARTEMIS-4406] - connection router LocalCache persisted entry tracking is not thread safe
[ARTEMIS-4410] - Openwire prefetched messages can be out of order after failover to an exclusive queue
[ARTEMIS-4415] - org.apache.activemq.artemis.tests.integration.server.LVQTest#testMultipleMessages fails intermittently
[ARTEMIS-4417] - AbstractJournalStorageManager storeKeyValuePair + deleteKeyValuePair are not thread safe
[ARTEMIS-4418] - openwire lastDeliveredSequenceId depends on message order, it should not
[ARTEMIS-4421] - Page Counters are not working before rebuild is done
[ARTEMIS-4424] - "AMQ212025: did not connect the cluster" when bootstrapping a static cluster
[ARTEMIS-4427] - MDB reusing Thread is using wrong transactionTimeout
[ARTEMIS-4431] - AMQP federated address consumer not applying hops annotation correctly

New Feature:
[ARTEMIS-3057] - Provide alternative to max-disk-usage to measure by remaining disk free
[ARTEMIS-4159] - Support duplicate cache size configuration per address
[ARTEMIS-4372] - Move CLI framework to picocli and implement auto-complete
[ARTEMIS-4375] - JLine3 integration
[ARTEMIS-4384] - CLI method to verify topology on all the nodes (cluster verify)
[ARTEMIS-4385] - Expand queue stat to other members of the topology
[ARTEMIS-4419] - Add broker federation support to the AMQP broker connection feature-set

Improvement:
[ARTEMIS-966] - MQTT Session States do not survive a reboot
[ARTEMIS-4349] - Replace Guava cache with Caffeine
[ARTEMIS-4368] - ensure predictable order of subjects for accurate logging
[ARTEMIS-4378] - Federation, ignore address policy when using pull consumer connection
[ARTEMIS-4391] - tests: rework AssertionLoggerHandler
[ARTEMIS-4396] - Make address/queue "internal" property durable
[ARTEMIS-4398] - Support configuring Database with Broker Properties
[ARTEMIS-4401] - Improving Paging & JDBC Performance
[ARTEMIS-4404] - Update the artemis-docker readme.md with minor clarification on building local distribution
[ARTEMIS-4408] - Update docker-run.sh for overriding etc folder after instance creation
[ARTEMIS-4411] - Change log level from ActiveMQRALogger.instantiatingDestination to DEBUG
[ARTEMIS-4428] - Expand default loggers configuration

Apache Spark 3.5.0
Features and Enhancements:
    SSE: DSNode to update result with names to make each value identifiable by labels (only Graphite and TestData.

Bug Fixes:
LDAP: Fix user disabling.

Apache Spark 3.5.0
Highlights

Scala and Go client support in Spark Connect SPARK-42554 SPARK-43351
PyTorch-based distributed ML Support for Spark Connect SPARK-42471
Structured Streaming support for Spark Connect in Python and Scala SPARK-42938
Pandas API support for the Python Spark Connect Client SPARK-42497
Introduce Arrow Python UDFs SPARK-40307
Support Python user-defined table functions SPARK-43798
Migrate PySpark errors onto error classes SPARK-42986
PySpark Test Framework SPARK-44042
Add support for Datasketches HllSketch SPARK-16484
Built-in SQL Function Improvement SPARK-41231
IDENTIFIER clause SPARK-43205
Add SQL functions into Scala, Python and R API SPARK-43907
Add named argument support for SQL functions SPARK-43922
Avoid unnecessary task rerun on decommissioned executor lost if shuffle data migrated SPARK-41469
Distributed ML <> spark connect SPARK-42471
DeepSpeed Distributor SPARK-44264
Implement changelog checkpointing for RocksDB state store SPARK-43421
Introduce watermark propagation among operators SPARK-42376
Introduce dropDuplicatesWithinWatermark SPARK-42931
RocksDB state store provider memory management enhancements SPARK-43311

Spark Connect

Refactoring of the sql module into sql and sql-api to produce a minimum set of dependencies that can be shared between the Scala Spark Connect client and Spark and avoids pulling all of the Spark transitive dependencies. SPARK-44273
Introducing the Scala client for Spark Connect SPARK-42554
Pandas API support for the Python Spark Connect Client SPARK-42497
PyTorch-based distributed ML Support for Spark Connect SPARK-42471
Structured Streaming support for Spark Connect in Python and Scala SPARK-42938
Initial version of the Go client SPARK-43351
Lot’s of compatibility improvements between Spark native and the Spark Connect clients across Python and Scala
Improved debugability and request handling for client applications (asynchronous processing, retries, long-lived queries)

Spark SQL
Features

Add metadata column file block start and length SPARK-42423
Support positional parameters in Scala/Java sql() SPARK-44066
Add named parameter support in parser for function calls SPARK-43922
Support SELECT DEFAULT with ORDER BY, LIMIT, OFFSET for INSERT source relation SPARK-43071
Add SQL grammar for PARTITION BY and ORDER BY clause after TABLE arguments for TVF calls SPARK-44503
Include column default values in DESCRIBE and SHOW CREATE TABLE output SPARK-42123
Add optional pattern for Catalog.listCatalogs SPARK-43792
Add optional pattern for Catalog.listDatabases SPARK-43881
Callback when ready for execution SPARK-44145
Support Insert By Name statement SPARK-42750
Add call_function for Scala API SPARK-44131
Stable derived column aliases SPARK-40822
Support general constant expressions as CREATE/REPLACE TABLE OPTIONS values SPARK-43529
Support subqueries with correlation through INTERSECT/EXCEPT SPARK-36124
IDENTIFIER clause SPARK-43205
ANSI MODE: Conv should return an error if the internal conversion overflows SPARK-42427

Functions

Add support for Datasketches HllSketch SPARK-16484
Support the CBC mode by aes_encrypt()/aes_decrypt() SPARK-43038
Support TABLE argument parser rule for TableValuedFunction SPARK-44200
Implement bitmap functions SPARK-44154
Add the try_aes_decrypt() function SPARK-42701
array_insert should fail with 0 index SPARK-43011
Add to_varchar alias for to_char SPARK-43815
High-order function: array_compact implementation SPARK-41235
Add analyzer support of named arguments for built-in functions SPARK-44059
Add NULLs for INSERTs with user-specified lists of fewer columns than the target table SPARK-42521
Adds support for aes_encrypt IVs and AAD SPARK-43290
DECODE function returns wrong results when passed NULL SPARK-41668
Support udf ‘luhn_check’ SPARK-42191
Support implicit lateral column alias resolution on Aggregate SPARK-41631
Support implicit lateral column alias in queries with Window SPARK-42217
Add 3-args function aliases DATE_ADD and DATE_DIFF SPARK-43492

Data Sources

Char/Varchar Support for JDBC Catalog SPARK-42904
Support Get SQL Keywords Dynamically Thru JDBC API and TVF SPARK-43119
DataSource V2: Handle MERGE commands for delta-based sources SPARK-43885
DataSource V2: Handle MERGE commands for group-based sources SPARK-43963
DataSource V2: Handle UPDATE commands for group-based sources SPARK-43975
DataSource V2: Allow representing updates as deletes and inserts SPARK-43775
Allow jdbc dialects to override the query used to create a table SPARK-41516
SPJ: Support partially clustered distribution SPARK-42038
DSv2 allows CTAS/RTAS to reserve schema nullability SPARK-43390
Add spark.sql.files.maxPartitionNum SPARK-44021
Handle UPDATE commands for delta-based sources SPARK-43324
Allow V2 writes to indicate advisory shuffle partition size SPARK-42779
Support lz4raw compression codec for Parquet SPARK-43273
Avro: writing complex unions SPARK-25050
Speed up Timestamp type inference with user-provided format in JSON/CSV data source SPARK-39280
Avro to Support custom decimal type backed by Long SPARK-43901
Avoid shuffle in Storage-Partitioned Join when partition keys mismatch, but join expressions are compatible SPARK-41413
Change binary to unsupported dataType in CSV format SPARK-42237
Allow Avro to convert union type to SQL with field name stable with type SPARK-43333
Speed up Timestamp type inference with legacy format in JSON/CSV data source SPARK-39281

Query Optimization

Subexpression elimination support shortcut expression SPARK-42815
Improve join stats estimation if one side can keep uniqueness SPARK-39851
Introduce the group limit of Window for rank-based filter to optimize top-k computation SPARK-37099
Fix behavior of null IN (empty list) in optimization rules SPARK-44431
Infer and push down window limit through window if partitionSpec is empty SPARK-41171
Remove the outer join if they are all distinct aggregate functions SPARK-42583
Collapse two adjacent windows with the same partition/order in subquery SPARK-42525
Push down limit through Python UDFs SPARK-42115
Optimize the order of filtering predicates SPARK-40045

Code Generation and Query Execution

Runtime filter should supports multi level shuffle join side as filter creation side SPARK-41674
Codegen Support for HiveSimpleUDF SPARK-42052
Codegen Support for HiveGenericUDF SPARK-42051
Codegen Support for build side outer shuffled hash join SPARK-44060
Implement code generation for to_csv function (StructsToCsv) SPARK-42169
Make AQE support InMemoryTableScanExec SPARK-42101
Support left outer join build left or right outer join build right in shuffled hash join SPARK-36612
Respect RequiresDistributionAndOrdering in CTAS/RTAS SPARK-43088
Coalesce buckets in join applied on broadcast join stream side SPARK-43107
Set nullable correctly on coalesced join key in full outer USING join SPARK-44251
Fix IN subquery ListQuery nullability SPARK-43413

Other Notable Changes

Set nullable correctly for keys in USING joins SPARK-43718
Fix COUNT(*) is null bug in correlated scalar subquery SPARK-43156
Dataframe.joinWith outer-join should return a null value for unmatched row SPARK-37829
Automatically rename conflicting metadata columns SPARK-42683
Document the Spark SQL error classes in user-facing documentation SPARK-42706

PySpark
Features

Support positional parameters in Python sql() SPARK-44140
Support parameterized SQL by sql() SPARK-41666
Support Python user-defined table functions SPARK-43797
Support to set Python executable for UDF and pandas function APIs in workers during runtime SPARK-43574
Add DataFrame.offset to PySpark SPARK-43213
Implement dir() in pyspark.sql.dataframe.DataFrame to include columns SPARK-43270
Add option to use large variable width vectors for arrow UDF operations SPARK-39979
Make mapInPandas / mapInArrow support barrier mode execution SPARK-42896
Add JobTag APIs to PySpark SparkContext SPARK-44194
Support for Python UDTF to analyze in Python SPARK-44380
Expose TimestampNTZType in pyspark.sql.types SPARK-43759
Support nested timestamp type SPARK-43545
Support UserDefinedType in createDataFrame from pandas DataFrame and toPandas [SPARK-43817]SPARK-43702
Add descriptor binary option to Pyspark Protobuf API SPARK-43799
Accept generics tuple as typing hints of Pandas UDF SPARK-43886
Add array_prepend function SPARK-41233
Add assertDataFrameEqual util function SPARK-44061
Support arrow-optimized Python UDTFs SPARK-43964
Allow custom precision for fp approx equality SPARK-44217
Make assertSchemaEqual API public SPARK-44216
Support fill_value for ps.Series SPARK-42094
Support struct type in createDataFrame from pandas DataFrame SPARK-43473

Other Notable Changes

Add autocomplete support for df[ ] in pyspark.sql.dataframe.DataFrame [SPARK-43892]
Deprecate & remove the APIs that will be removed in pandas 2.0 [SPARK-42593]
Make Python the first tab for code examples - Spark SQL, DataFrames and Datasets Guide SPARK-42493
Updating remaining Spark documentation code examples to show Python by default SPARK-42642
Use deduplicated field names when creating Arrow RecordBatch [SPARK-41971]
Support duplicated field names in createDataFrame with pandas DataFrame [SPARK-43528]
Allow columns parameter when creating DataFrame with Series [SPARK-42194]

Core

Schedule mergeFinalize when push merge shuffleMapStage retry but no running tasks SPARK-40082
Introduce PartitionEvaluator for SQL operator execution SPARK-43061
Allow ShuffleDriverComponent to declare if shuffle data is reliably stored SPARK-42689
Add max attempts limitation for stages to avoid potential infinite retry SPARK-42577
Support log level configuration with static Spark conf SPARK-43782
Optimize PercentileHeap SPARK-42528
Add reason argument to TaskScheduler.cancelTasks SPARK-42602
Avoid unnecessary task rerun on decommissioned executor lost if shuffle data migrated SPARK-41469
Fixing accumulator undercount in the case of the retry task with rdd cache SPARK-41497
Use RocksDB for spark.history.store.hybridStore.diskBackend by default SPARK-42277
Support spark.kubernetes.setSubmitTimeInDriver SPARK-43014
NonFateSharingCache wrapper for Guava Cache SPARK-43300
Improve the performance of MapOutputTracker.updateMapOutput SPARK-43043
Allowing apps to control whether their metadata gets saved in the db by the External Shuffle Service SPARK-43179
Port executor failure tracker from Spark on YARN to K8s SPARK-41210
Parameterize the max number of attempts for driver props fetcher in KubernetesExecutorBackend SPARK-42764
Add SPARK_DRIVER_POD_IP env variable to executor pods SPARK-42769
Mounts the hadoop config map on the executor pod SPARK-43504

Structured Streaming

Add support for tracking pinned blocks memory usage for RocksDB state store SPARK-43120
Add RocksDB state store provider memory management enhancements SPARK-43311
Introduce dropDuplicatesWithinWatermark SPARK-42931
Introduce a new callback onQueryIdle() to StreamingQueryListener SPARK-43183
Add option to skip commit coordinator as part of StreamingWrite API for DSv2 sources/sinks SPARK-42968
Introduce a new callback “onQueryIdle” to StreamingQueryListener SPARK-43183
Implement Changelog based Checkpointing for RocksDB State Store Provider SPARK-43421
Add support for WRITE_FLUSH_BYTES for RocksDB used in streaming stateful operators SPARK-42792
Add support for setting max_write_buffer_number and write_buffer_size for RocksDB used in streaming SPARK-42819
RocksDB StateStore lock acquisition should happen after getting input iterator from inputRDD SPARK-42566
Introduce watermark propagation among operators SPARK-42376
Cleanup orphan sst and log files in RocksDB checkpoint directory SPARK-42353
Expand QueryTerminatedEvent to contain error class if it exists in exception SPARK-43482

Support Distributed Training of Functions Using Deepspeed SPARK-44264
Base interfaces of sparkML for spark3.5: estimator/transformer/model/evaluator SPARK-43516
Make MLv2 (ML on spark connect) supports pandas >= 2.0 SPARK-43783
Update MLv2 Transformer interfaces SPARK-43516
New pyspark ML logistic regression estimator implemented on top of distributor SPARK-43097
Add Classifier.getNumClasses back SPARK-42526
Write a Deepspeed Distributed Learning Class DeepspeedTorchDistributor SPARK-44264
Basic saving / loading implementation for ML on spark connect SPARK-43981
Improve logistic regression model saving SPARK-43097
Implement pipeline estimator for ML on spark connect SPARK-43982
Implement cross validator estimator SPARK-43983
Implement classification evaluator SPARK-44250
Make PyTorch Distributor compatible with Spark Connect SPARK-42993

Add a Spark UI page for Spark Connect SPARK-44394
Support Heap Histogram column in Executors tab SPARK-44153
Show error message on UI for each failed query SPARK-44367
Display Add/Remove Time of Executors on Executors Tab SPARK-44309

Elasticsearch 8.10.1
Bug Fixes
Aggregations:
Use long in Centroid count #99491 (issue: #80153)

Infra/Core:
Fix deadlock between Cache.put and Cache.invalidateAll #99480 (issue: #99326)

Infra/Node Lifecycle:
Fork computation in TransportGetShutdownStatusAction #99490 (issue: #99487)

Search:
Fix PIT when resolving with deleted indices #99281

Grafana 10.1.0
Flame graph improvements
Generally available in all editions of Grafana
We’ve added four new features to the Flame graph visualization:

Sandwich view: You can now show a sandwich view of any symbol in the flame graph. Sandwich view shows all the callers on the top and all the callees of the symbol on the bottom. This is useful when you want to see the context of a symbol.
Switching color scheme: You can now switch color scheme between a color gradient based on the relative value of a symbol or by the package name of a symbol.
Switching symbol name alignment: Symbols with long names may be hard to differentiate if they have the same prefix. This new option allows you to align the text to the left or right so you can see the part of the symbol name that’s important.
Improved navigation: You can highlight a symbol or enable sandwich view for a symbol from the table. Also, a new status bar on top of the flame graph displays which views are enabled.

Jenkins 2.423
Move node monitoring option to app bar. (pull 8381)
Symbols display in breadcrumbs now. (issue 71983)
Developer: make branding an extension via SimplePageDecorator. (pull 8462)

Kibana 8.10.1
Bug Fixes
Dashboard:
Fixes content editor flyout footer (#165907).

Elastic Security:
For the Elastic Security 8.10.1 release information, refer to Elastic Security Solution Release Notes.

Fleet:
Show snapshot version in agent upgrade modal and allow custom values (#165978).

Observability:
Fix(slo): Use comma-separarted list of source index for transform (#166294).

Presentation:
Fixes air-gapped enviroment hitting 400 error when loading fonts for layer (#165986).

Kubernetes 1.28.2
API Change

Fixed a bug where CEL expressions in CRD validation rules would incorrectly compute a high estimated cost for functions that return strings, lists or maps. The incorrect cost was evident when the result of a function was used in subsequent operations.
Mark Job onPodConditions as optional in pod failure policy

Feature

Kubernetes is now built with Go 1.20.8

Bug or Regression

Fix OpenAPI v3 not being cleaned up after deleting APIServices
Fix a 1.28 regression in scheduler: a pod with concurrent events could incorrectly get moved to the unschedulable queue where it could got stuck until the next periodic purging after 5 minutes if there was no other event for it.
Fix a concurrent map access in TopologyCache's HasPopulatedHints method.
Fixed a 1.26 regression scheduling bug by ensuring that preemption is skipped when a PreFilter plugin returns UnschedulableAndUnresolvable
Fixed a 1.27 scheduling regression that PostFilter plugin may not function if previous PreFilter plugins return Skip
Fixed a 1.28 regression around restarting init containers in the right order relative to normal containers
Fixed a regression in default 1.27 configurations in kube-apiserver: fixed the AggregatedDiscoveryEndpoint feature (beta in 1.27+) to successfully fetch discovery information from aggregated API servers that do not check Acceptheaders when serving the /apis endpoint
Fixes a 1.28 regression handling negative index json patches
Fixes a bug where images pinned by the container runtime can be garbage collected by kubelet.
Ignore context canceled from validate and mutate webhook
Kubeadm: fix nil pointer when etcd member is already removed

Logstash 8.10.1
No user-facing changes in Logstash core and plugins.

Nodejs 20.7.0
Notable Changes:
- src: support multiple --env-file declarations
- crypto: update root certificates to NSS 3.93
- deps: upgrade npm to 10.1.0
- (SEMVER-MINOR) deps: upgrade npm to 10.0.0
- doc: move and rename loaders section
- doc: add release key for Ulises Gascon
- (SEMVER-MINOR) lib: add api to detect whether source-maps are enabled
- src,permission: add multiple allow-fs-* flags
- (SEMVER-MINOR) test_runner: expose location of tests

PostgreSQL 16
Performance Improvements

PostgreSQL 16 improves the performance of existing PostgreSQL functionality through new query planner optimizations. In this latest release, the query planner can parallelize FULL and RIGHT joins, generate better optimized plans for queries that use aggregate functions with a DISTINCT or ORDER BY clause, utilize incremental sorts for SELECT DISTINCT queries, and optimize window functions so they execute more efficiently. It also improves RIGHT and OUTER "anti-joins", which enables users to identify rows not present in a joined table.
This release includes improvements for bulk loading using COPY in both single and concurrent operations, with tests showing up to a 300% performance improvement in some cases. PostgreSQL 16 adds support for load balancing in clients that use libpq, and improvements to vacuum strategy that reduce the necessity of full-table freezes. Additionally, PostgreSQL 16 introduces CPU acceleration using SIMD in both x86 and ARM architectures, resulting in performance gains when processing ASCII and JSON strings, and performing array and subtransaction searches.

Logical Replication

Logical replication lets users stream data to other PostgreSQL instances or subscribers that can interpret the PostgreSQL logical replication protocol. In PostgreSQL 16, users can perform logical replication from a standby instance, meaning a standby can publish logical changes to other servers. This provides developers with new workload distribution options, for example, using a standby rather than the busier primary to logically replicate changes to downstream systems.
Additionally, there are several performance improvements in PostgreSQL 16 to logical replication. Subscribers can now apply large transactions using parallel workers. For tables that do not have a primary key, subscribers can use B-tree indexes instead of sequential scans to find rows. Under certain conditions, users can also speed up initial table synchronization using the binary format.
There are several access control improvements to logical replication in PostgreSQL 16, including the new predefined role pg_create_subscription, which grants users the ability to create new logical subscriptions. Finally, this release begins adding support for bidirectional logical replication, introducing functionality to replicate data between two tables from different publishers.

Developer Experience

PostgreSQL 16 adds more syntax from the SQL/JSON standard, including constructors and predicates such as JSON_ARRAY(), JSON_ARRAYAGG(), and IS JSON. This release also introduces the ability to use underscores for thousands separators (e.g. 5_432_000) and non-decimal integer literals, such as 0x1538, 0o12470, and 0b1010100111000.
Developers using PostgreSQL 16 also benefit from new commands in psql. This includes \bind, which allows users to prepare parameterized queries and use \bind to substitute the variables (e.g SELECT $1::int + $2::int \bind 1 2 \g).
PostgreSQL 16 improves general support for text collations, which provide rules for how text is sorted. PostgreSQL 16 builds with ICU support by default, determines the default ICU locale from the environment, and allows users to define custom ICU collation rules.

Monitoring

A key aspect of tuning the performance of database workloads is understanding the impact of your I/O operations on your system. PostgreSQL 16 introduces pg_stat_io, a new source of key I/O metrics for granular analysis of I/O access patterns.
Additionally, this release adds a new field to the pg_stat_all_tables view that records a timestamp representing when a table or index was last scanned. PostgreSQL 16 also makes auto_explain more readable by logging values passed into parameterized statements, and improves the accuracy of the query tracking algorithm used by pg_stat_statements and pg_stat_activity.

Access Control & Security

PostgreSQL 16 provides finer-grained options for access control and enhances other security features. The release improves management of pg_hba.conf and pg_ident.conf files, including allowing regular expression matching for user and database names and include directives for external configuration files.
This release adds several security-oriented client connection parameters, including require_auth, which allows clients to specify which authentication parameters they are willing to accept from a server, and sslrootcert="system", which indicates that PostgreSQL should use the trusted certificate authority (CA) store provided by the client's operating system. Additionally, the release adds support for Kerberos credential delegation, allowing extensions such as postgres_fdw and dblink to use authenticated credentials to connect to trusted services.

RabbitMQ 3.11.23
Core Server Bug Fixes

High consumer churn with reused consumer tag on quorum queues could result in some messages not being delivered
after a period of time.

This did not affect environments where consumer churn does not exist or where it does but consumer tags vary.

Three environment variables, LOG_BASE, MNESIA_BASE, CONFIG_FILE, were not picked up when set in
rabbitmq-env-conf.bat on Windows.
Avoids a potential exception when autoheal partition handling process was initiated.

OpenUpdate - September 14, 2023

Stay Informed

This week, read about:

Android 14 Blocks All Modification of System Certificates, Even As Root.
Welcome New Repositories for AlmaLinux OS: Testing and Synergy.
Ubuntu 23.10 to Feature Experimental TPM-backed Full Disk Encryption.
Apache Cassandra 5.0 Is Coming: Here’s Why the People Who Built It Are Fired Up.
With Version 117, Firefox Finally Speaks Chrome's Translation Language.

Key Security, Maintenance, and Features Releases

Security Based Updates

Redis 7.2.1
Upgrade urgency SECURITY: See security fixes below.
Security Fixes:
(CVE-2023-41053) Redis does not correctly identify keys accessed by SORT_RO and, as a result, may grant users executing this command access to keys that are not explicitly authorized by the ACL configuration.

Bug Fixes
Fix crashes when joining a node to an existing 7.0 Redis Cluster
Correct request_policy and response_policy command tips on for some admin /configuration commands.

Non-Security Based Updates

Elasticsearch 8.9.2
Bug Fixes:
Data streams: Avoid lifecycle NPE in the data stream lifecycle usage API #98260
Geo: Fix mvt error when returning partial results #98765 (issue: #98730)
Ingest Node: Revert "Add mappings for enrich fields" #98683

Grafana 10.0.5
Features and Enhancements;
SSE: DSNode to update result with names to make each value identifiable by labels (only Graphite and TestData.

Bug fixes:
LDAP: Fix user disabling.

Jenkins 2.422

Prevent incorrect readResolve implementations from breaking agent label parsing.
Update several buttons and menus to replace YahooUI in more locations.
List plugins in deterministic order to improve diagnosability of plugin linkage errors.
Add telemetry collecting basic information about the security configuration.
Update Turkish localization for the new job page.
Upgrade to Winstone 6.13 to include Jetty 10.0.16.
Developer: Initialize default view slightly earlier in the initialization process.

Kibana 8.9.2
Enhancements
Fleet:
- Adds the configuration setting xpack.fleet.packageVerification.gpgKeyPath as an environment variable in the Kibana container (#163783).

Bug Fixes
Dashboard:
- Fixes missing state on short URLs could be lost on an alias match redirect (#163658).
- Fixes Download CSV returning no data when panel has custom time range outside the time range of the global time picker (#163887).
- Fixes Dashboard getting stuck at loading in Kibana when Controls is used and mapping changed from integer to keyword (#163529).

Elastic Security:
- For the Elastic Security 8.9.2 release information, refer to Elastic Security Solution Release Notes.

Lens & Visualizations:
- Allow removing temporary data view from event annotation group in Lens (#163976).

Machine Learning:
- Anomaly detection wizard: ensure custom URLs test functionality works as expected (#165055).
- Fixes anomaly detection module manifest queries for Kibana sample data sets, so cold and frozen tiers are not queried (#164332).

Management:
- Transforms: Fixes privileges check (#163687).

Operations:
- Fixes an issue where Kibana did not start on CentOS/RHEL 7 (#165151).

Reporting:
- Allow custom roles to use image reporting in Dashboard

Logstash 8.9.2
No user facing changes.

Node.js 20.6.1
Changes:
- esm: fix loading of CJS modules from ESM
- benchmark: add benchmarks for the test_runner
- benchmark: add pm startup benchmark
- child_process: harden against prototype pollution
- deps: V8: cherry-pick 93275031284c
- deps: update simdutf to 3.2.17
- deps: update googletest to 7e33b6a (
- deps: update zlib to 1.2.13.1-motley-526382e
- deps: update undici to 5.23.0
- deps: update googletest to c875c4e
- deps: update ada to 2.6.0
- deps: upgrade npm to 9.8.1
- deps: update zlib to 1.2.13.1-motley-61dc0bd
- deps: V8: cherry-pick 9f4b7699f68e
- deps: V8: cherry-pick c1a54d5ffcd1
- deps: update googletest to cc36671
- diagnostics_channel: fix last subscriber removal
- doc: add rluvaton to collaborators
- doc: add print results for examples in WebStreams
- doc: fix Type notation in webstreams
- doc: fix name of the flag in initialize() docs
- doc: make the NODE_VERSION_IS_RELEASE revert clear
- doc: update process.binding deprecation text
- doc: update with latest security release
- doc: add description for --port flag of node inspect
- doc: add missing period
- doc: add ESM examples in http.md
- doc: detailed description of keystrokes Ctrl-Y and Meta-Y
- doc: add "type" to test runner event details
- doc: reserve 118 for Electron 27
- doc: clarify use of process.env in worker threads on Windows
- doc: remove v14 mention
- doc: drop github actions check in sec release process
- doc: improved joinDuplicateHeaders definition
- doc: fix second parameter name of events.addAbortListener
- doc: add new reporter events to custom reporter examples
- doc: run license-builder
- doc: change duration to duration_ms on test documentation
- doc: improve requireHostHeader
- doc: add ver of 18.x where Node-api 9 is supported
- doc: include experimental features assessment
- doc: add new TSC members
- doc: refactor node-api support matrix
- doc: declare path on example of async_hooks.executionAsyncId()
- doc: remove the . in the end to reduce confusing
- doc: nodejs-social over nodejs/tweet
- doc: expand on squashing and rebasing to land a PR
- esm: fix globalPreload warning
- esm: unflag import.meta.resolve
- esm: import.meta.resolve exact module not found errors should return
- esm: protect ERR_UNSUPPORTED_DIR_IMPORT against prototype pollution
- esm: add initialize hook, integrate with register
- esm: fix typo parentUrl -> parentURL
- esm: unflag Module.register and allow nested loader import()
- esm: add back globalPreload tests and fix failing ones
- events: remove weak listener for event target
- fs: fix readdir recursive sync & callback
- fs: mention URL in NUL character error message
- fs: make mkdtemp accept buffers and URL
- fs: remove redundant nullCheck
- http: start connections checking interval on listen
- (SEMVER-MINOR) inspector: open add SymbolDispose
- lib: fix MIME overmatch in data URLs
- lib: fix to add resolve() before return at Blob.stream()'s source.pull()
- lib: remove invalid parameter to toASCII
- lib,permission: drop repl autocomplete when pm enabled
- meta: bump github/codeql-action from 2.20.1 to 2.21.2
- meta: bump step-security/harden-runner from 2.4.1 to 2.5.0
- meta: bump actions/setup-node from 3.6.0 to 3.7.0
- meta: bump actions/setup-python from 4.6.1 to 4.7.0
- meta: add mailmap entry for atlowChemi
- module: make CJS load from ESM loader
- module: ensure successful import returns the same result
- module: implement register utility
- node-api: avoid macro redefinition (
- permission: move PrintTree into unnamed namespace
- permission: fix data types in PrintTree
- readline: add paste bracket mode
- sea: add support for V8 bytecode-only caching
- src: use effective cppgc wrapper id to deduce non-cppgc id
- src: add built-in .env file support
- src: remove duplicated code in GenerateSingleExecutableBlob()
- src: refactor vector writing in snapshot builder
- src: add ability to overload fast api functions
- src: remove redundant code for uv_handle_type
- src: modernize use-equals-default
- src: avoid string copy in BuiltinLoader::GetBuiltinIds
- src: fix callback_queue.h missing header
- src: cast v8::Object::GetInternalField() return value to v8::Value
- src: do not pass user input to format string
- src: remove ContextEmbedderIndex::kBindingDataStoreIndex
- src: use ARES_SUCCESS instead of 0
- src: save the performance milestone time origin in the AliasedArray
- src: support snapshot in single executable applications
- src: remove unnecessary temporary creation
- src: fix nullptr access on realm
- src: remove OnScopeLeaveImpl's move assignment overload
- src: use string_view for utf-8 string creation
- src,permission: restrict by default when pm enabled
- src,tools: initialize cppgc
- stream: improve WebStreams performance
- stream: implement ReadableStream.from
- test: use tmpdir.resolve()
- test: use tmpdir.resolve()
- test: use tmpdir.resolve() in fs tests
- test: use tmpdir.resolve() in fs tests
- test: fix assertion message in test_async.c
- test: refactor test-esm-loader-hooks for easier debugging
- test: add tmpdir.resolve()
- test: document fixtures.fileURL()
- test: reduce flakiness of test-esm-loader-hooks
- test: stabilize the inspector-open-dispose test
- test: print instruction for creating missing snapshot in assertSnapshot
- test: add tmpdir.fileURL()
- test: use spawn and spawnPromisified instead of exec
- test: refactor test-node-output-errors
- test: use fixtures.fileURL when appropriate
- test: validate error code rather than message
- test: fix snapshot tests when cwd contains spaces or backslashes
- test: order common.mjs in ASCII order
- test: fix some assumptions in tests
- test: improve internal/worker/io.js coverage
- test: fix es-module/test-esm-initialization
- test: validate host with commas on url.parse
- test: delete test-net-bytes-per-incoming-chunk-overhead
- test: skip experimental test with pointer compression
- test: fix flaky test-string-decode.js on x86
- test_runner: dont set exit code on todo tests
- test_runner: fix todo and only in spec reporter
- test_runner: unwrap error message in TAP reporter
- test_runner: add __proto__ null
- test_runner: fix async callback in describe not awaited
- test_runner: fix test_runner test:fail event type
- test_runner: call abort on test finish
- tls: fix bugs of double TLS
- tools: update lint-md-dependencies
- tools: use spec reporter in actions
- tools: use @reporters/github when running in github
- tools: add @reporters/github to tools
- tools: update eslint to 8.47.0
- tools: update lint-md-dependencies to rollup@3.27.2
- tools: limit the number of auto start CIs
- tools: update eslint to 8.46.0
- tools: update lint-md-dependencies to rollup@3.27.0
- tools: update lint-md-dependencies to rollup@3.26.3
- tools: update lint-md-dependencies to @rollup/plugin-commonjs@25.0.3
- tools: update eslint to 8.45.0
- typings: update JSDoc for cwd in child_process
- typings: sync JSDoc with the actual implementation
- url: overload canParse V8 fast api method
- url: fix isURL detection by checking path
- url: ensure getter access do not mutate observable symbols
- url: reduce pathToFileURL cpp calls
- util: use primordials.ArrayPrototypeIndexOf instead of mutable method
- watch: decrease debounce rate
- watch: use debounce instead of throttle

Prometheus 2.47.0
This version is compiled with Go 1.21.0.
[FEATURE] Web: Add OpenTelemetry (OTLP) Ingestion endpoint.
[FEATURE] Scraping: Optionally limit detail on dropped targets, to save memory.
[ENHANCEMENT] TSDB: Write head chunks to disk in the background to reduce blocking.
[ENHANCEMENT] PromQL: Speed up aggregate and function queries.
[ENHANCEMENT] PromQL: More efficient evaluation of query with timestamp().
[ENHANCEMENT] API: Faster streaming of Labels to JSON.
[ENHANCEMENT] Agent: Memory pooling optimisation.
[ENHANCEMENT] TSDB: Prevent storage space leaks due to terminated snapshots on shutdown.
[ENHANCEMENT] Histograms: Refactoring and optimisations.
[ENHANCEMENT] Histograms: Add histogram_stdvar and histogram_stddev functions.
[ENHANCEMENT] Remote-write: add http.resend_count tracing attribute.
[ENHANCEMENT] TSDB: Support native histograms in snapshot on shutdown.
[BUGFIX] TSDB/Agent: ensure that new series get written to WAL on rollback.
[BUGFIX] Scraping: fix infinite loop on exemplar in protobuf format.

Sonatype Nexus Repository 3.60.0
Bug Fixes
NEXUS-4014: Fixed the previously reported Repair - Reconcile component database from blob store task issue. The bug caused the task to soft-delete the blob .properties and .bytes files for NuGet v2 proxy and hosted repositories. It also failed to restore the desired content for RubyGems, NuGet v2 (proxy or hosted), or P2 repositories; however, there was no soft deletion associated with RubyGems or P2 repositories.

NEXUS-39918: Clarified search restrictions in high availability environments to explain that searches cannot begin with a special character followed by a wildcard. Attempts to perform such seareches will now result in appropriate descriptive messaging.

NEXUS-39825: NuGet v3 search now returns the complete list of component versions even when the component name has a dot after a digit.

NEXUS-38670: Improved Apt upload performance and speed.

NEXUS-37537: The lastDownloaded attribute for hosted Helm assets now updates as expected in deployments using PostgreSQL or H2.

NEXUS-37024: The Global Webhook capability with Audit Type now works as expected.

Strimzi 0.37.0
This release contains the following new features and improvements:

The StableConnectIdentites feature gate moves to a beta stage. By default, StrimziPodSets are used for Kafka Connect and Kafka Mirror Maker 2. If needed, StableConnectIdentites can be disabled in the feature gates configuration in the Cluster Operator.
Support for the ppc64le platform
Added version fields to the Kafka custom resource status to track installation and upgrade state
Support for infinite auto-restarts of Kafka Connect and Kafka Mirror Maker 2 connectors

It also has several notable changes, deprecations, and removals:
Removed support for OpenTracing:

The tracing.type: jaeger configuration, in KafkaConnect, KafkaMirrorMaker, KafkaMirrorMaker2 and KafkaBridge resources, is not supported anymore.
The OpenTelemetry-based tracing is the only available by using tracing.type: opentelemetry.
The default behavior of the Kafka Connect connector auto-restart has changed. When the auto-restart feature is enabled in KafkaConnector or KafkaMirrorMaker2 custom resources, it will now continue to restart the connectors indefinitely rather than stopping after 7 restarts, as previously.
If you want to use the original behavior, use the .spec.autoRestart.maxRestarts option to configure the maximum number of restarts.

The automatic configuration of Cruise Control CPU capacity has been changed in this release:

There are three ways to configure Cruise Control CPU capacity values:
.spec.cruiseControl.brokerCapacity (for all brokers)
.spec.cruiseControl.brokerCapacity.overrides (per broker)
Kafka resource requests and limits (for all brokers).
The precedence of which Cruise Control CPU capacity configuration is used has been changed.
In previous Strimzi versions, the Kafka resource limit (if set) took precedence, regardless if any other CPU configurations were set.

For example:
- (1) Kafka resource limits
- (2) .spec.cruiseControl.brokerCapacity.overrides
- (3) .spec.cruiseControl.brokerCapacity

This previous behavior was identified as a bug and was fixed in this Strimzi release.

Going forward, the brokerCapacity overrides per broker take top precedence, then general brokerCapacity configuration, and then the Kafka resource requests, then the Kafka resource limits.

For example:
- (1) .spec.cruiseControl.brokerCapacity.overrides
- (2) .spec.cruiseControl.brokerCapacity
- (3) Kafka resource requests
- (4) Kafka resource limits

When none of Cruise Control CPU capacity configurations mentioned above are configured, CPU capacity will be set to 1.
as any override value configured in the .spec.cruiseControl section of the Kafka custom resource.

OpenUpdate - September 7, 2023

Stay Informed

This week, read about:

LibreOffice 7.6 Arrives: Open Source Stalwart is Showing Its Maturity.
Welcome New Repositories for AlmaLinux OS: Testing and Synergy.
Unit 1.31.0 Released.
AMD Shares The Technical Details of Technology Powering Innovative Confidential Computing Leadership Cloud Offerings.

Key Security, Maintenance, and Features Releases

Security Based Updates

Gitlab 16.3.1
Fixed (1 change):
- [Geo: Resync direct upload object stored artifacts] **GitLab Enterprise Edition**

Security (11 changes):
- [Add authorization checks to import status endpoint]
- [Update commonmarker to 0.23.10]
- [Remove DAST secret variables when URL is updated]
- [Maintainer can leak sentry token by changing the configured URL]
- [Service account users are external by default]
- [Additional permission check when editing label]
- [Fix ReDOS in bulk_imports endpoint params]
- [Prevent namespace level banned users from accessing API]
- [Check prohibit_outer_forks in fork relationship api]
- [Prevent traversal for `path` parameter in refs/switch endpoint]
- [Gitaly keyset pager when pagination none only with tree view]

Security Based Updates

Docker Compose Engine 2.21.0
Features:
- Support for multi-document YAML files.
- Experimental support for loading remote Compose files from Git repos with include.

Fixes:
- Fix for incorrect proxy variables during build.
- Fix for truncated container logs.
- Fix for "no such service" errors when using include and profiles.
- Fix for .env overrides when using include.

Grafana 10.1.1
Features and Enhancements:
- Loki: Remove distinct operation.
- Whitelabeling: Add a config option to hide the Grafana edition from the footer.
- Alerting: Optimize rule details page data fetching.
- Alerting: Optimize external Loki queries.

Bug Fixes:
- Alerting: Limit redis pool size to 5 and make configurable.
- Elasticsearch: Fix respecting of precision in geo hash grid.
- Dashboard: Fix Variable Dropdown to Enforce Minimum One Selection when 'All' Option is Configured.
- Chore: Fix Random Walk scenario for Grafana DS.
- AuthProxy: Fix user retrieval through cache.
- Alerting: Fix auto-completion snippets for KV properties.
- Alerting: Fix incorrect timing meta information for policy.
- Alerting: Add new Recording Rule button when the list is empty.
- Drawer: Clicking a Select arrow within a Drawer no longer causes it to close.
- Logs: Fix log samples not present with empty first frame.
- Alerting: Fix Recording Rule QueryEditor builder view.
- Transforms: Catch errors while running transforms.
- Dashboard: Fix version restore.
- Logs: Fix permalinks not scrolling into view.
- SqlDataSources: Update metricFindQuery to pass on scopedVars to templateSrv.
- Rendering: Fix dashboard screenshot.
- Loki: Fix validation of step values to also allow e.g. ms values.
- Dashboard: Fix repeated row panel placement with larger number of rows.
- CodeEditor: Correctly fires onChange handler.
- Drawer: Fix scrolling drawer content on Safari.
- Alerting: Remove dump wrapper for yaml config.
- Alerting: Always invalidate the AM config after mutation.
- Slug: Combine various slugify fixes for special character handling.
- Logs: Fix displaying the wrong field as body.
- Alerting: Fix "see graph button" for cloud rules.

Jenkins 2.421
- Add a nicer 404 error page.
- Add appearance system configuration page.
- Optimize performance of label parsing.
- Fix invalid CSS which caused some buttons to become invisible on hover.
- Message no longer appears twice when the agentLog option is used.

MongoDB 7.0.1
Security:
SERVER-78723: Resharding a QE collection fails because of __safeContent__
SERVER-78830: Add count of CSFLE and QE Collections to serverStatus
SERVER-79641: Mirrored read should attach encryptionInformation from the original command

Sharding:
SERVER-62987: Wrong replication logic on refreshes on secondary nodes
SERVER-67529: Resharding silently skips documents with all MaxKey values for their fields under the new shard key pattern
SERVER-78913: Make the periods of query sampling periodic jobs configurable at runtime

Query:
SERVER-80256: QueryPlannerAnalysis::explodeForSort should not assume that index scans produce disjoint results

Internals:
SERVER-71627: Refreshed cached collection route info will severely block all client request when a cluster with 1 million chunks
SERVER-73866: Re-enable agg_merge_when_not_matched_insert.js in config_fuzzer passthrough suites
SERVER-74701: Add checksum verification for blackduck installer
SERVER-75120: libunwind stacktrace issues with --dbg=on on arm64
SERVER-76299: Report writeConflicts in serverStatus on secondaries
SERVER-76339: Increase ShardedClusterFixture's timeout when starting/stopping balancer
SERVER-76433: Copy search_view.js test from 5.0 to all later branches
SERVER-77029: Set syncdelay in TestOplogTruncation before starting the checkpoint thread
SERVER-77183: $project followed by $group gives incorrect results sometimes
SERVER-77223: dbcheck_detects_data_corruption.js needs to wait for primary to log healthlog entry
SERVER-77382: Null embedded metaField for creating a time-series collection leads to invalid BSON index spec
SERVER-77823: Pseudocode for throughput probing
SERVER-78095: Relax the assertion checking for update_multifield_multiupdate.js FSM workload
SERVER-78217: Renaming view return wrong error on sharded cluster (2nd attempt)
SERVER-78369: ignoreUnknownIndexOptions doesn't account for the 'weights' index field
SERVER-78498: Make the balancer failpoint smarter
SERVER-78525: Update jstests/noPassthrough/metadata_size_estimate.js to use a smaller document size
SERVER-78696: Only clear shard filtering metadata before releasing the critical section in collmod participants
SERVER-78769: The asynchronous stop sequence of the Balancer may survive the shutdown of the mongod (and raise false memory leak notifications).
SERVER-78813: Commit point propagation fails indefinitely with exhaust cursors with null lastCommitted optime
SERVER-78862: Fix serialization of nested $elemMatch's
SERVER-78950: Use sequential time series bucket IDs when possible
SERVER-79021: Update Boost's entry in README.third_party.md to 1.79.0
SERVER-79022: Update ASIO's Git hash in README.third_party.md
SERVER-79023: Update C-Ares' entry in README.third_party.md to 1.19.1
SERVER-79033: Image collection invalidation for missing namespace during initial sync always attempts upsert
SERVER-79082: Make analyzeShardKey tests not assert number of orphaned documents <= total number of documents
SERVER-79103: Core dumps are not generated if stopping balancer fails
SERVER-79126: Pin pyyaml in another place
SERVER-79138: Fix data race in AuthorizationSessionTest fixture
SERVER-79236: Server cannot start in standalone if there are cluster parameters
SERVER-79252: Add the system-perf bootstrap file to the task Files section
SERVER-79261: Add logging to ping monitor
SERVER-79316: [7.0] Do not run packager on dynamically linked variants
SERVER-79357: CheckMetadataConsistency is not reading chunks with snapshot read concern
SERVER-79370: Throughput probing statistics not always updated correctly
SERVER-79372: Fix incorrect assertion about number of cursors opened
SERVER-79382: Reset bucket OID counter when encountering a collision
SERVER-79397: Fix and test logic to internally retry time series inserts on OID collision
SERVER-79447: The balancer stop sequence may cause the config server to crash on step down
SERVER-79509: Add testing of transitional FCVs with removeShard and transitionToDedicatedConfigServer
SERVER-79515: Update task generator
SERVER-79607: ShardRegistry shutdown should not wait indefinitely on outstanding network requests
SERVER-79609: Fix findAndModify_upsert.js test to accept StaleConfig error
SERVER-79651: Only use two node replicasets in initial sync performance tests
SERVER-79777: Increase the diff window for the sample size in sample_rate_sharded.js
SERVER-79885: Oplog fetching getMore should not set null lastKnownCommittedOpTime if it is not using exhaust cursors
SERVER-79937: Avoid majority reads within the BalancerDefragmentationPolicy
SERVER-79944: Make analyze_shard_key.js not assert that the number of sampled queries observed via analyzeShardKey and $listSampledQueries is non-decreasing
SERVER-79950: Fix commitPreparedTransaction to not be interruptible in commitSplitTxn and reacquireTicket
SERVER-79981: resize_tickets.js fails in Fixed Concurrent Transactions test suite
SERVER-80153: UBsan core dumps are not being uploaded properly
SERVER-80183: Remove operationTime check from store_retryable_find_and_modify_images_in_side_collection.js
SERVER-80207: Use 4-byte counter for tracking time series bucket direct writes
WT-10714: Select an explicitly labeled perf distro for performance tests
WT-11202: Remove the connection level operation_timeout_ms configuration
WT-11221: Python tests fails due to unexpected "Eviction took more than 1 minute" warning in standard output
WT-11312: Fix incorrect flag check for accurate force eviction stat
WT-11359: Update spinlock tasks to limit disk usage
WT-11419: Increment cc_pages_removed when detecting a deleted page to remove

PHP Interpreter 8.2.10
CLI:
Fixed bug GH-11716 (cli server crashes on SIGINT when compiled with ZEND_RC_DEBUG=1).
Fixed bug GH-10964 (Improve man page about the built-in server).

Date:
Fixed bug GH-11416 (Crash with DatePeriod when uninitialised objects are passed in).

Core:
Fixed strerror_r detection at configuration time.
Fixed trait typed properties using a DNF type not being correctly bound.
Fixed trait property types not being arena allocated if copied from an internal trait.
Fixed deep copy of property DNF type during lazy class load.
Fixed memory freeing of DNF types for non arena allocated types.

DOM:
Fix DOMEntity field getter bugs.
Fix incorrect attribute existence check in DOMElement::setAttributeNodeNS.
Fix DOMCharacterData::replaceWith() with itself.
Fix empty argument cases for DOMParentNode methods.
Fixed bug GH-11791 (Wrong default value of DOMDocument::xmlStandalone).
Fix json_encode result on DOMDocument.
Fix manually calling __construct() on DOM classes.
Fixed bug GH-11830 (ParentNode methods should perform their checks upfront).
Fix viable next sibling search for replaceWith.
Fix segfault when DOMParentNode::prepend() is called when the child disappears.

FFI:
Fix leaking definitions when using FFI::cdef()->new(...).

Hash:
Fix use-of-uninitialized-value in hash_pbkdf2(), fix missing $options parameter in signature.

MySQLnd:
Fixed bug GH-11440 (authentication to a sha256_password account fails over SSL).
Fixed bug GH-11438 (mysqlnd fails to authenticate with sha256_password accounts using passwords longer than 19 characters).
Fixed bug GH-11550 (MySQL Statement has a empty query result when the response field has changed, also Segmentation fault).
Fixed invalid error message "Malformed packet" when connection is dropped.

Opcache:
Fixed bug GH-11715 (opcache.interned_strings_buffer either has no effect or opcache_get_status() / phpinfo() is wrong).
Avoid adding an unnecessary read-lock when loading script from shm if restart is in progress.

PCNTL:
Revert behaviour of receiving SIGCHLD signals back to the behaviour before 8.1.22.

SPL:
Fixed bug #81992 (SplFixedArray::setSize() causes use-after-free).

Standard:
Prevent int overflow on $decimals in number_format.
Fixed bug GH-11870 (Fix off-by-one bug when truncating tempnam prefix) (athos-ribeiro)

Ceph 16.2.14
backport PR #39607
blk/kernel: Fix error code mapping in KernelDevice::read
blk/KernelDevice: Modify the rotational and discard check log message
build: Remove ceph-libboost* packages in install-deps
ceph-volume: fix a bug in get_lvm_fast_allocs() (batch)
ceph-volume: fix batch refactor issue
ceph-volume: fix drive-group issue that expects the batch_args to be a string
ceph-volume: quick fix in zap.py
ceph-volume: set lvm membership for mpath type devices
ceph_test_rados_api_watch_notify: extend Watch3Timeout test
ceph_volume: support encrypted volumes for lvm new-db/new-wal/migrate commands
cephadm: eliminate duplication of sections
cephadm: mount host /etc/hosts for daemon containers in podman deployments
cephadm: reschedule haproxy from an offline host
cephadm: using ip instead of short hostname for prometheus urls
cephfs-top: check the minimum compatible python version
cephfs-top: dump values to stdout and -d [--delay] option fix
cephfs-top: navigate to home screen when no fs
cephfs-top: Some fixes in choose_field() for sorting
client: clear the suid/sgid in fallocate path
client: do not dump mds twice in Inode::dump()
client: do not send metrics until the MDS rank is ready
client: force sending cap revoke ack always
client: only wait for write MDS OPs when unmounting
client: trigger to flush the buffer when making snapshot
client: use deep-copy when setting permission during make_request
client: wait rename to finish
cls/queue: use larger read chunks in queue_list_entries
common/crc32c_aarch64: fix crc32c unittest failed on aarch64
common/TrackedOp: fix osd reboot optracker coredump
common: notify all when max backlog reached in OutputDataSocket
common: Use double instead of long double to improve performance
Consider setting “bulk” autoscale pool flag when automatically creating a data pool for CephFS
debian: install cephfs-mirror systemd unit files and man page
do not evict clients if OSDs are laggy
doc/cephadm: Revert “doc/cephadm: update about disabling logging to journald for quincy”
doc/cephfs: edit fs-volumes.rst (1 of x)
doc/cephfs: explain cephfs data and metadata set
doc/cephfs: fix prompts in fs-volumes.rst
doc/cephfs: line-edit “Mirroring Module”
doc/cephfs: rectify prompts in fs-volumes.rst
doc/cephfs: repairing inaccessible FSes
doc/dev/encoding.txt: update per std::optional
doc/glossary: update bluestore entry
doc/mgr: edit “leaderboard” in telemetry.rst
doc/mgr: update prompts in prometheus.rst
doc/rados/operations: Acting Set question
doc/rados/operations: Fix erasure-code-jerasure.rst fix
doc/rados/ops: edit user-management.rst (3 of x)
doc/rados: edit balancer.rst
doc/rados: edit bluestore-config-ref.rst (1 of x)
doc/rados: edit bluestore-config-ref.rst (2 of x)
doc/rados: edit data-placement.rst
doc/rados: edit devices.rst
doc/rados: edit filestore-config-ref.rst
doc/rados: edit stretch-mode procedure
doc/rados: edit stretch-mode.rst
doc/rados: edit stretch-mode.rst
doc/rados: edit user-management (2 of x)
doc/rados: fix link in common.rst
doc/rados: line-edit devices.rst
doc/rados: m-config-ref: edit “background”
doc/rados: stretch-mode.rst (other commands)
doc/rados: stretch-mode: stretch cluster issues
doc/radosgw: explain multisite dynamic sharding
doc/radosgw: rabbitmq - push-endpoint edit
doc/start/os-recommendations: drop 4.14 kernel and reword guidance
doc/start: edit first 150 lines of documenting-ceph
doc/start: fix “Planet Ceph” link
doc/start: KRBD feature flag support note
doc/start: rewrite intro paragraph
doc: add link to “documenting ceph” to index.rst
doc: Add missing ceph command in documentation section REPLACING A…
doc: deprecate the cache tiering
doc: document the relevance of mds_namespace mount option
doc: explain cephfs mirroring peer_add step in detail
doc: Update jerasure.org references
doc: update multisite doc
doc: Use ceph osd crush tree command to display weight set weights
kv/RocksDBStore: Add CompactOnDeletion support
kv/RocksDBStore: cumulative backport for rm_range_keys and around (
kv/RocksDBStore: don’t use real wholespace iterator for prefixed access
librados: aio operate functions can set times
librbd/managed_lock/GetLockerRequest: Fix no valid lockers case
librbd: avoid decrementing iterator before first element
librbd: avoid object map corruption in snapshots taken under I/O
librbd: don’t wait for a watch in send_acquire_lock() if client is blocklisted
librbd: localize snap_remove op for mirror snapshots
librbd: remove previous incomplete primary snapshot after successfully creating a new one
log: writes to stderr (pipe) may not be atomic
MDS imported_inodes metric is not updated
mds: adjust cap acquisition throttles
mds: allow unlink from lost+found directory
mds: display sane hex value (0x0) for empty feature bit
mds: do not send split_realms for CEPH_SNAP_OP_UPDATE msg
mds: do not take the ino which has been used
mds: fix cpu_profiler asok crash
mds: fix stray evaluation using scrub and introduce new option
mds: Fix the linkmerge assert check
mds: force replay sessionmap version
mds: make num_fwd and num_retry to __u32
mds: MDLog::_recovery_thread: handle the errors gracefully
mds: rdlock_path_xlock_dentry supports returning auth target inode
mds: record and dump last tid for trimming completed requests (or flushes)
mds: skip forwarding request if the session were removed
mds: update mdlog perf counters during replay
mds: wait for unlink operation to finish
mds: wait reintegrate to finish when unlinking
mgr/cephadm: Adding --storage.tsdb.retention.size prometheus option
mgr/cephadm: don’t try to write client/os tuning profiles to known offline hosts
mgr/cephadm: support for miscellaneous config files for daemons
mgr/dashboard: allow PUT in CORS
mgr/dashboard: API docs UI does not work with Angular dev server
mgr/dashboard: expose more grafana configs in service form
mgr/dashboard: Fix broken Fedora image URL
mgr/dashboard: Fix rbd snapshot creation
mgr/dashboard: fix the rbd mirroring configure check
mgr/dashboard: move cephadm e2e cleanup to jenkins job config
mgr/dashboard: rbd-mirror force promotion
mgr/dashboard: skip Create OSDs step in Cluster expansion
mgr/dashboard: SSO error: AttributeError: ‘str’ object has no attribute ‘decode’
mgr/nfs: disallow non-existent paths when creating export
mgr/orchestrator: fix device size in orch device ls output
mgr/rbd_support: fixes related to recover from rados client blocklisting
mgr/snap_schedule: add debug log for paths failing snapshot creation
mgr/snap_schedule: catch all exceptions for cli
mgr/volumes: avoid returning -ESHUTDOWN back to cli
mgr: store names of modules that register RADOS clients in the MgrMap
MgrMonitor: batch commit OSDMap and MgrMap mutations
mon/ConfigMonitor: update crush_location from osd entity
mon/MDSMonitor: batch last_metadata update with pending
mon/MDSMonitor: check fscid in pending exists in current
mon/MDSMonitor: do not propose on error in prepare_update
mon/MDSMonitor: ignore extraneous up:boot messages
mon/MonClient: before complete auth with error, reopen session
mon: avoid exception when setting require-osd-release more than 2 versions up
mon: block osd pool mksnap for fs pools
Monitor: forward report command to leader
orchestrator: add --no-destroy arg to ceph orch osd rm
os/bluestore: allocator’s cumulative backport
os/bluestore: allow ‘fit_to_fast’ selector for single-volume osd
os/bluestore: cumulative bluefs backport
os/bluestore: don’t need separate variable to mark hits when lookup oid
os/bluestore: fix spillover alert
os/bluestore: proper override rocksdb::WritableFile::Allocate
os/bluestore: report min_alloc_size through “ceph osd metadata”
osd/OSDCap: allow rbd.metadata_list method under rbd-read-only profile
OSD: Fix check_past_interval_bounds()
pybind/argparse: blocklist ip validation
pybind/mgr/pg_autoscaler: Reorderd if statement for the func: _maybe_adjust
pybind: drop GIL during library callouts
python-common: drive_selection: fix KeyError when osdspec_affinity is not set
qa/rgw: add POOL_APP_NOT_ENABLED to log-ignorelist
qa/suites/rados: remove rook coverage from the rados suite
qa/suites/rbd: install qemu-utils in addition to qemu-block-extra on Ubuntu
qa/suites/upgrade/octopus-x: skip TestClsRbd.mirror_snapshot test
qa: check each fs for health
qa: data-scan/journal-tool do not output debugging in upstream testing
qa: fix cephfs-mirror unwinding and ‘fs volume create/rm’ order
qa: mirror tests should cleanup fs during unwind
qa: run scrub post file system recovery
qa: test_simple failure
qa: use parallel gzip for compressing logs
qa: wait for MDSMonitor tick to replace daemons
radosgw-admin: try reshard even if bucket is resharding
rbd-mirror: fix image replayer shut down description on force promote
rbd-mirror: fix race preventing local image deletion
rgw/rados: check_quota() uses real bucket owner
rgw/s3: dump Message field in Error response even if empty
rgw: avoid string_view to temporary in RGWBulkUploadOp
rgw: fix consistency bug with OLH objects
rgw: LDAP fix resource leak with wrong credentials
rgw: under fips & openssl 3.x allow md5 usage in select rgw ops
src/valgrind.supp: Adding know leaks unrelated to ceph
src/valgrind.supp: Adding know leaks unrelated to ceph
test: correct osd pool default size
test: monitor thrasher wait until quorum
tests: remove pubsub tests from multisite
tools/ceph-dencoder: Fix incorrect type define for trash_watcher
tools/ceph-kvstore-tool: fix segfaults when repair the rocksdb
tools/cephfs-data-scan: support for multi-datapool
vstart: check mgr status after starting mgr
Wip nitzan fixing few rados/test.sh
qa: add subvolume option flavors

Ansible AWX 23.0.0
- Revert "Improve performance for awx cli export
- Fixed typos
- Schedule rruleset fix related #13446
- Update python-tss-sdk dependency
- Fix UI_NEXT build process broken
- Fixed task and web docs
- Fix ui-next build step file path issue
- Added required epoc time field for Splunk HEC Event Receiver
- Fix edit constructed inventory hanging loading state
- Add location for locales in nginx config
- Update cryptography for CVE-2023-38325
- AAP-10891 Apply AWX_TASK_ENV when performing credential plugin lookups
- Enforce mutually exclusive options in credential module of the collection
- Clarify that the license module requires fetching subs prior
- Fix default redis url to pass check in redis-py>4.4
- Fix typo in description of scm_update_on_launch
- Fix CVE-2023-40267
- Touchup of PR body checks
- Hop nodes for k8s

OpenUpdate - August 31, 2023

Stay Informed

This week, read about:

Linus Torvalds Couldn't Find An Excuse To Hold Back Linux 6.5, So Here It Is.
Welcome New Repositories for AlmaLinux OS: Testing and Synergy.
SUSE To Flip Back Into Private Ownership After Just Two-and-a-Bit Years.
CVE-2023-40217.

Key Security, Maintenance, and Features Releases

Security Based Updates

Updates to the OpenLogic CentOS Repository

OpenLogic’s Enterprise Linux Team has recently published the following updates:

CVE-2022-38177 and CVE-2022-38178
CentOS 8
bind-9.11.26-6_ol001.el8

We recommend that you update your CentOS 8 systems to protect against this vulnerability.

As usual, please ensure that you test these updates before deploying to production.

If you don't currently have CentOS repo access, please reach out to your Perforce/OpenLogic salesperson … you may already be entitled to access with your existing support contract!

Kubernetes 1.28.1
This release contains changes that address the following vulnerabilities:

CVE-2023-3955: Insufficient input sanitization on Windows nodes leads to privilege escalation

A security issue was discovered in Kubernetes where a user that can create pods on Windows nodes may be able to escalate to admin privileges on those nodes. Kubernetes clusters are only affected if they include Windows nodes.

Affected Versions:

kubelet <= v1.28.0
kubelet <= v1.27.4
kubelet <= v1.26.7
kubelet <= v1.25.12
kubelet <= v1.24.16

Fixed Versions:

kubelet v1.28.1
kubelet v1.27.5
kubelet v1.26.8
kubelet v1.25.13
kubelet v1.24.17

CVSS Rating: High (8.8) CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H

CVE-2023-3676: Insufficient input sanitization on Windows nodes leads to privilege escalation

Affected Versions:

kubelet <= v1.28.0
kubelet <= v1.27.4
kubelet <= v1.26.7
kubelet <= v1.25.12
kubelet <= v1.24.16

Fixed Versions:

kubelet v1.28.1
kubelet v1.27.5
kubelet v1.26.8
kubelet v1.25.13
kubelet v1.24.17

CVSS Rating: High (8.8) CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H

Non-Security Updates

Angular 16.2.2
*Allow safeUrl for ngSrc in NgOptimizedImage
*enforce a minimum version to be used when a library uses input transform
*guard the jasmine hooks
*Ensure canceledNavigationResolution: 'computed' works on first page

Apache Tomcat 10.1.13
Catalina:
Fix: If an application or library sets both a non-500 error code and the jakarta.servlet.error.exception request attribute, use the provided error code during error page processing rather than assuming an error code of 500.
Fix: Update code comments and Tomcat output to use MiB for 1024 * 1024 bytes and KiB for 1024 bytes rather than MB and kB.
Fix: Avoid protocol relative redirects in FORM authentication.

Web applications:
Fix: Documentation. Update documentation to use MiB for 1024 * 1024 bytes and KiB for 1024 bytes rather than MB and kB.

Other:
Add: Improvements to Chinese translations.
Add: Improvements to French translations.
Add: Improvements to Japanese translations by tak7iji.

Grafana 10.1.0

Sandwich view: You can now show a sandwich view of any symbol in the flame graph. Sandwich view shows all the callers on the top and all the callees of the symbol on the bottom. This is useful when you want to see the context of a symbol.
Switching color scheme: You can now switch color scheme between a color gradient based on the relative value of a symbol or by the package name of a symbol.
Switching symbol name alignment: Symbols with long names may be hard to differentiate if they have the same prefix. This new option allows you to align the text to the left or right so you can see the part of the symbol name that’s important.
Improved navigation: You can highlight a symbol or enable sandwich view for a symbol from the table. Also, a new status bar on top of the flame graph displays which views are enabled.
Many more improvements

Jenkins 2.420

Move plugins page title into sidebar so that plugins app bar is at the top of the page.
Remove eval call in hudsonbehavior.js.
Update Turkish localizations for the job configuration page.
Refresh link design.
Display a notice when plugin updates are available or when there are no plugins installed.
Update the design of the content blocks. (
Remove the unnecessary hashelp class additions from hudsonbehaviour.js.
Hide administrative monitors icons/popup in the header of Manage Jenkins, as they're shown directly on the page.
The plain text console log will still be printed even if some console annotations are corrupt.
Fix link to job in the message informing administrators of trigger computations that run for an unusually long time.
Deprecate findAncestor and findAncestorClass in hudsonbehaviour.js.

RabbitMQ 3.12.4
Core Server
Bug Fixes:

Consumer churn on quorum queues could result in some messages not being delivered
Management Plugin

Bug Fixes:

Quorum queue replica management operations over the HTTP API now can be
disabled. This can be useful in environments where replica management
is done by the platform team and tooling, and should not be exposed to
cluster users.

Federation Plugin
Bug Fixes:

Queue federation links that connected quorum queues could get stuck
(stop transferring messages even when there were no other consumers on the upstream).

LDAP AuthN/AuthZ Backend Plugin
Bug Fixes:

LDAP plugin did not interpolate values with non-ASCII characters correctly.

OpenUpdate - August 24, 2023

Stay Informed

This week, read about:

Google Introduces First Quantum Resilient FIDO2 Security Key Implementation.
Announcement of LibreOffice 7.6 Community.
TP-Link Smart Bulbs Can Let Hackers Steal Your WiFi Password.
We're In The OWASP-Makes-List-Of-Security-Bug-Types Phase with LLM Chatbots.

Key Security, Maintenance, and Features Releases

Security Based Updates

Apache Cassandra 3.11.16
* Moved jflex from runtime to build dependencies (CASSANDRA-18664)
* Fix CAST function for float to decimal (CASSANDRA-18647)
* Suppress CVE-2022-45688 (CASSANDRA-18643)
* Remove unrepaired SSTables from garbage collection when only_purge_repaired_tombstones is true (CASSANDRA-14204)
* Wait for live endpoints in gossip waiting to settle (CASSANDRA-18543)
* Fix error message handling when trying to use CLUSTERING ORDER with non-clustering column (CASSANDRA-17818
* Add keyspace and table name to exception message during ColumnSubselection deserialization (CASSANDRA-18346)
* Remove unnecessary String.format invocation in QueryProcessor when getting a prepared statement from cache (CASSANDRA-17202)

Merged from 3.0:
* Fix Requires for Java for RPM package (CASSANDRA-18751)
* Fix CQLSH online help topic link (CASSANDRA-17534)
* Remove unused suppressions (CASSANDRA-18724)
* Upgrade OWASP to 8.3.1 (CASSANDRA-18650)
* Suppress CVE-2023-34462 (CASSANDRA-18649)
* Add support for AWS Ec2 IMDSv2 (CASSANDRA-16555)
* Suppress CVE-2023-35116 (CASSANDRA-18630)
* Pass taskId from CompactionTask to system.compaction_history (CASSANDRA-12183)
* Suppress CVE-2023-34455, CVE-2023-34454, CVE-2023-34453 (CASSANDRA-18608)
* Backport CASSANDRA-10508: Remove hard-coded SSL cipher suites (CASSANDRA-18575)
* Suppress CVE-2023-2976 (CASSANDRA-18562)
* Remove dh_python use in Debian packaging (CASSANDRA-18558)
* Pass down all contact points to driver for cassandra-stress (CASSANDRA-18025)
* Validate the existence of a datacenter in nodetool rebuild (CASSANDRA-14319)
* Suppress CVE-2023-2251 (CASSANDRA-18497)

Non-Security Based Updates

Nginx 1.25.2
* Feature: path MTU discovery when using HTTP/3.
* Feature: TLS_AES_128_CCM_SHA256 cipher suite support when using HTTP/3.
* Change: now nginx uses appname "nginx" when loading OpenSSL configuration.
* Change: now nginx does not try to load OpenSSL configuration if the --with-openssl option was used to built OpenSSL and the OPENSSL_CONF environment variable is not set.
* Bugfix: in the $body_bytes_sent variable when using HTTP/3.
* Bugfix: in HTTP/3.

Angular 16.2.1
* Fix: Apply named outlets to children empty paths not appearing in the URL.

Elasticsearch 8.9.1
Fixes:
Aggregations

GlobalAggregator should call rewrite() before createWeight()

Cluster Coordination

Improve exception handling in Coordinator#publish

EQL

Backport fix for async missing events and re-enable the feature

ILM+SLM

Ignore the total_shards_per_node setting on searchable snapshots in frozen
Migrate to data tiers routing configures correct default for mounted indices Infra/Core
Fix APM trace start time

Infra/Logging

Add Configuration to PatternLayout

Machine Learning

Fix failure processing Question Answering model output where the input has been spanned over multiple sequences

UnmappedFieldFetcher should ignore nested fields

Grafana 9.5.8
Features and Enhancements:
GenericOAuth: Set sub as auth id.

Bug Fixes:
DataSourceProxy: Fix url validation error handling

Kibana 8.9.1
Fixes:
APM

Fixes flame graph rendering on the transaction detail page.
Check if documents are missing span.name.
Fixes transaction action menu for Trace Explorer and dependency operations.

Canvas

Fixes embeddables not rendering in Canvas.

Discover

Fixes grid styles to enable better content wrapping.
Fixes search sessions using temporary data views.
Make share links and search session information shorter for temporary data views.

Fleet

Fixes for query error on Agents list in the UI.
Remove duplicate path being pushed to package archive.

Management

Resolves potential errors present in v8.9.0 with data views that contain field filters that have been edited.

Uptime

Fixes Monitor not found 404 message display.

Kubernetes 1.28
UPGRADE NOTES

Action required for the custom scheduler plugin developers. Here's the breaking change in EnqueueExtension in the scheduling framework. The EventsToRegister in EnqueueExtension changed the return value from ClusterEvent to ClusterEventWithHint. ClusterEventWithHint allows each plugin to filter out more useless events via the callback function named QueueingHintFn. When the scheduling queue receives a cluster event, before moving each Pod from unschedulable pod pool to activeQ/backoffQ, it will call QueueingHintFn of plugins that rejected each Pod in the previous scheduling cycle. Depending on the value returned from QueueingHintFn, the scheduling queue changes how it queues each Pod:
- if more than one QueueingHintFn returns QueueImmediately, it queues Pod to activeQ.
- If no QueueingHintFn returns QueueImmediately and more than one plugin returns QueueAfterBackoff, it queues Pod to backoffQ if Pod is backing off, or to activeQ if Pod's backoff has already finished.
- If all QueueingHintFn return QueueSkip, it puts this pod back to the unschedulable pod pool

Having appropriate QueueingHintFn contributes to reducing useless retries and thus improves the overall scheduler's performance.

How can I migrate?

For backward compatibility, nil QueueingHintFn is treated as always returning QueueAfterBackoff. So, if you want to just keep the existing behavior, you can register ClusterEventWithHint with no QueueingHintFn in it. But, registering appropriate QueueingHintFn is, of course, better from a scheduling performance perspective.

CephFS volume plugin (kubernetes.io/cephfs) has been deprecated in this release and will be removed in a subsequent release. The alternative is to use the CephFS CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes cluster.
Deprecated support for CSI migration of Ceph RBD volumes. Users who were relying on Kubernetes' ability to migrate to an out-of-tree storage driver should complete that migration before the support for it is removed.
RBD volume plugin (kubernetes.io/rbd) has been deprecated in this release and will be removed in a subsequent release. Alternative is to use RBD CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes Cluster.

FIXES
Deprecation:

Changed kubectl version default output to be identical to what kubectl version --short printed, and removed --short flag entirely.
Kube-controller-manager deprecate --volume-host-cidr-denylist and --volume-host-allow-local-loopback flags.
Kubelet: The --azure-container-registry-config flag has been deprecated and will be removed in a future release, please use --image-credential-provider-config and --image-credential-provider-bin-dir to setup acr credential provider instead.
Removed tracking annotation from validation and defaulting.
Removed withdrawn feature NetworkPolicyStatus.
The deprecated flag --lock-object-namespace and --lock-object-name have been removed from kube-scheduler. Please use --leader-elect-resource-namespace and --leader-elect-resource-name or ComponentConfig instead to configure those parameters.
KMSv1 is deprecated and will only receive security updates going forward. Use KMSv2 instead. In a future release, Set --feature-gates=KMSv1=true to use the deprecated KMSv1 feature.

API Change:

A CDIDevice field is included in the Device Plugin's ContainerAllocateResponse. This field maps to the CDIDevice field in the CRI protocol.
ACTION_REQUIRED When an Indexed Job has a number of completions higher than 10^5 and parallelism higher than 10^4, and a big number of Indexes fail, Kubernetes might not be able to track the termination of the Job. Kubernetes now emits a warning, at Job creation, when the Job manifest exceeds both of these limits.
Added ServedVersions field to StorageVersion API.
Added IP mode field to loadbalancer status ingress.
Added podReplacementPolicy and terminating field to job api.
Added a new namespaceParamRef field to admissionregistration.k8s.io/v1alpha1.ValidatingAdmissionPolicy.
Added a warning that TLS 1.3 ciphers are not configurable.
Added error handling for seccomp localhost configurations that do not properly set a localhostProfile.
Added fields reason and fieldPath into CRD validation rules to allow users to specify reason and field path when validation failed.
Added namespace access support to the CEL expressions of ValidatingAdmissionPolicy via a namespaceObject variable with expressions.
Added new CRDValidationRatcheting alpha feature. During a PATCH or UPDATE Validation Ratcheting discards errors thrown by unchanged portions of the resource from most OpenAPI schema validations.
Added new annotation batch.kubernetes.io/cronjob-scheduled-timestamp to Job objects scheduled from CronJobs.
Added new config option delayCacheUntilActive to KubeSchedulerConfiguration that can provide a tradeoff between memory efficiency and scheduling speed when their leadership is updated in kube-scheduler
Changed how KMS v2 encryption at rest can generate data encryption keys. When you enable the KMSv2KDF feature gate (off by default), KMS v2 uses a key derivation function to generate single use data encryption keys from a secret seed combined with some random data. This eliminates the need for a counter based nonce while avoiding nonce collision concerns associated with AES-GCM's 12 byte nonce.
Exposed rest.DefaultServerUrlFor function.
Extended the Job API for alpha version of BackoffLimitPerIndex.
Graduated AdmissionWebhookMatchCondition feature to beta.
If using cgroups v2, then the cgroup aware OOM killer will be enabled for container cgroups via memory.oom.group . This causes processes within the cgroup to be treated as a unit and killed simultaneously in the event of an OOM kill on any process in the cgroup.
In the API Priority and Fairness feature, priority levels that are exempt from limitation can now be given a nominal and a lendable concurrency and their dispatching borrows from the concurrency limits of the other priority levels.
Indexed Job pods now have the pod completion index set as a pod label.
Kube-proxy: added --logging-format flag to support structured logging.
NodeVolumeLimits implement the PreFilter extension point for skipping the Filter phase if the Pod doesn't use volumes with limits.
PersistentVolumes have a new LastPhaseTransitionTime field which holds a timestamp of when the volume last transitioned its phase.
Pods which set hostNetwork: true and declare ports, get the hostPort field set automatically. Previously this would happen in the PodTemplate of a Deployment, DaemonSet or other workload API. Now hostPort will only be set when an actual Pod is being created. If this presents a problem, setting the feature gate "DefaultHostNetworkHostPortsInPodTemplates" to true will revert this behavior. Please file a kubernetes bug if you need to do this.
Promoted API groups ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding to v1beta1.
Promoted the feature gate ValidtaingAdmissionPolicy to beta, and it is turned off by default.
Registered_metric_total, disabled_metric_total, hidden_metric_total & kubernetes_feature_enabled are promoted to BETA stability.
Removed resizeStatus enum from pvc.Status and replaced with AllocatedResourceStatus.
Removed WindowsHostProcessContainers feature-gate.
Revised the comment about the feature-gate level for PodFailurePolicy from alpha to beta.
StatefulSet pods now have the pod index set as a pod label statefulset.kubernetes.io/pod-index.
Support for proxying a request to a peer kube-apiserver if the local apiserver is not able to serve it due to version skew or in the case the requested api is disabled on the local apiserver
Supported BackoffLimitPerIndex in Jobs.
The IPTablesOwnershipCleanup feature (KEP-3178) is now GA; kubelet no longer creates the KUBE-MARK-DROP chain (which has been unused for several releases) or the KUBE-MARK-MASQ chain (which is now only created by kube-proxy).
The SelfSubjectReview API is promoted to authentication.k8s.io/v1 and the kubectl auth whoami command is GA.
The names of ResourceClaims generated from ResourceClaimTemplate are now generated. The base name is still <pod>-<claim name>, but a random suffix will avoid name collisions.
The new feature gate "SidecarContainers" is now available. This feature introduces sidecar containers, a new type of init container that starts before other containers but remains running for the full duration of the pod's lifecycle and will not block pod termination.
Updated the comment about the feature-gate level for PodFailurePolicy from alpha to beta.
client-go: Improved memory use of reflector caches when watching large numbers of objects which do not change frequently.
component-base/logs is now stricter about not applying configurations multiple times and will return an error when that is attempted. Can be overridden by binaries which need to do that.
kube-controller-manager: The LegacyServiceAccountTokenCleanUp feature gate is now available as alpha (off by default). When enabled, the legacy-service-account-token-cleaner controller loop removes service account token secrets that have not been used in the time specified by --legacy-service-account-token-clean-up-period (defaulting to one year), and are referenced from the .secrets list of a ServiceAccount object, and are not referenced from pods.
kube-scheduler component config (KubeSchedulerConfiguration) kubescheduler.config.k8s.io/v1beta2 is removed in v1.28. Migrate kube-scheduler configuration files to kubescheduler.config.k8s.io/v1.

Feature:

A ValidatingAdmissionPolicy now has its messageExpression field checked against resolved types.
Added '--concurrent-cron-job-syncs' flag for kube-controller-manager to set the number of workers for cron job controller.
Added '--concurrent-job-syncs' flag for kube-controller-manager to set the number of job controller workers.
Added --concurrency flag to configure the concurrency of kubectl diff execution, defaults to 1.
Added ConsistentListFromCache feature gate that allows apiserver to serve consistent lists from cache.
Added DisruptionTarget condition to the pod preempted by kubelet to make room for a critical pod.
Added apiserver_admission_match_condition_evaluation_seconds and apiserver_admission_match_condition_exclusions_total metrics.
Added a container image for kubectl at registry.k8s.io/kubectl across the same architectures as other images (linux/amd64 linux/arm64 linux/s390x linux/ppc64le)
Added a new command line argument --interactive to kubectl. The new command line argument lets a user confirm deletion requests per resource interactively.
Added a new feature gate, SchedulerQueueingHints (enabled by default). The new feature gate activates a framework for fine-grained filtering of events related to scheduler plugins. In this release, no default scheduling plugins make use of the hinting framework, so you should not expect any behavior changes.
Added full cgroup v2 swap support for both Limited and Unlimited swap.

When LimitedSwap is enabled the swap limit would be automatically calculated for Burstable QoS pods. For Best-Effort/Guaranteed QoS pods, swap would be disabled.

Containers with memory requests equal to their memory limits also won't have swap access, and it is a way to opt-out of swap for a single container.

The formula for the swap limit for Burstable QoS pods is: (<memory-request>/<node-memory-capacity>)*<node-swap-capacity>.

Support for cgroup v1 is removed.

Added handling for pods in podgc for PodReplacementPolicy or PodDisruption.
Added reason to metric attachdetach_controller_forced_detaches in the attach detach controller.
Added support for pod hostNetwork field selector
Added swap to stats to Summary API and Prometheus endpoints (stats/summary and /metrics/resource).
Added the implementation for PodRecreationPolicy to wait for the creation of pods once the existing ones are fully terminated.
Allow to monitor client-go DNS resolver latencies via rest_client_dns_resolution_duration_seconds Prometheus metric.
Apiserver adds two new metrics etcd_requests_total and etcd_request_errors_total that allow users to monitor requests to etcd storage, split by operation and resource type.
Bumped distroless-iptables to 0.2.6 based on Go 1.20.6.
Bumped metrics-server to v0.6.3.
CEL authorizer checks no longer raise runtime errors. Calls to "check" will always return a decision object and the authorization error (if any) can be accessed within expressions using the new decision methods "errored" and "error".
CRI: exposed commit memory bytes in container stats specific to Windows
Client-go now exposes two new metrics to monitor the client-go logic that generate http.Transports for the clients.
- rest_client_transport_cache_entries is a gauge metric with the number of existing entries in the internal cache
- rest_client_transport_create_calls_total is a counter that increments each time a new transport is created, storing the result of the operation needed to generate it: hit, miss or uncacheable.
Cloud controller manager's node controller now emits timing metrics for initial Node synchronization. These metrics measure the delay between the creation of a new Node and the node controller's initial management actions, such as removing the cloud provider taint. These metrics should be consulted when setting cloud controller manager's --concurrent-node-syncs flag.
Dynamic resource allocation: when a claim uses "wait for first consumer" allocation (the default), then it will now get deallocated after it was used by a pod. That ensures that the next pod isn't affected by previous scheduling decision and that resources are not kept allocated unless really needed. If keeping a claim allocated is desired, use "immediate allocation."
Enabled use of pods with volumes and user namespaces. The feature gate was renamed from UserNamespacesStatelessPodsSupport to UserNamespacesSupport.
External credential provider plugins will now have their standard error output logged by kubelet upon failures.
Faster scheduling when ResourceClaims are involved.
Fixed the alpha CloudDualStackNodeIPs feature.
Graduated the LegacyServiceAccountTokenTracking feature gate to GA. The usage of auto-generated secret-based service account token now produces warnings, and relevant Secrets are labeled with a last-used timestamp (label key kubernetes.io/legacy-token-last-used).
Graduated the ProbeTerminationGracePeriod feature gate to GA.
Hashing of KeyID in Logs

This release adds a feature to hash the KeyID values in the logs. The KeyID values are sensitive information that should not be exposed in plain text in the logs. By hashing the KeyID values, we can protect the confidentiality of the data while still being able to log the necessary information.

Implemented alpha support for a drop-in kubelet configuration directory.
In the course of admitting a single request, the ValidatingAdmissionPolicy plugin will perform no more than one authorization check per unique authorizer expression. All evaluations of identical authorizer expressions will produce the same decision.
Introduce support for CEL optionals (see CEL spec proposal 246). This feature will not be fully enabled until a future Kubernetes release (likely to be v1.29), but is added in v1.28 to enable safe rollback on downgrade.
Kube-controller-manager: the dynamic resource controller steps in when a pod got created such that the scheduler ignores it (i.e. spec.nodeName is set) and then takes care of triggering delayed resource claim allocation and/or reserving a claim for the pod.
Kube-proxy handles Terminating EndpointSlices conditions and enables zero downtime deployments for Services with ExternalTrafficPolicy=Local author: @andrewsykim
Kube-proxy service health returns http header X-Load-Balancing-Endpoint-Weight with number of local endpoints. The same information is still available in response body JSON payload.LocalEndpoints.
Kubelet: plugins for dynamic resource allocation may use the v1alpha3 API instead of v1alpha2 if they want to do prepare/unprepare operations in batches.
Kubelet: security of dynamic resource allocation was enhanced by limiting node access to those objects that are needed on the node.
Kubelet: un-deprecated --provider-id flag.
Kubernetes is now built with Go 1.20.4.
Kubernetes is now built with Go 1.20.5.
Kubernetes is now built with Go 1.20.6.
Metric scheduler_scheduler_goroutines is removed. Use scheduler_goroutines instead.
Migrated pkg/controller/endpoint to contextual logging.
Migrated pkg/scheduler/framework/preemption to use contextual logging.
Migrated pod-security-admission to use contextual logging.
Migrated controller functions to use contextual logging.
Migrated the Job controller (within kube-controller-manager) to use contextual logging.
Migrated the EndpointSlice and EndpointSliceMirroring controllers (within kube-controller-manager) to use contextual logging.
Migrated the certificate controller (within kube-controller-manager) to use contextual logging.
Migrated the noderesources scheduler plugin to use contextual logging.
Migrated the podtopologyspread scheduler plugins to use contextual logging.
Moved non-graceful node shutdown to GA.
New CEL Library functions to support Kubernetes Quantities.
New Metrics Added for Encryption Configuration Controller

This release adds new metrics to the Encryption Configuration Controller to help monitor the automatic reloading of encryption configuration. The new metrics include:

apiserver_encryption_config_controller_automatic_reload_failures_total: Total number of failed automatic reloads of encryption configuration.
apiserver_encryption_config_controller_automatic_reload_success_total: Total number of successful automatic reloads of encryption configuration.
apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds: Timestamp of the last successful or failed automatic reload of encryption configuration.

These metrics can be used to monitor the health of the Encryption Configuration Controller and to troubleshoot any issues that may arise during automatic reloading of encryption configuration.

New staging repo has been created for the EndpointSlice reconciler.
Promoted ServiceNodePortStaticSubrange feature gate to beta, and it will be enabled by default.
Promoted the following apiserver flowcontrol metrics to Beta:
- apiserver_flowcontrol_request_wait_duration_seconds
- apiserver_flowcontrol_current_executing_seats
- apiserver_flowcontrol_nominal_limit_seats
- apiserver_flowcontrol_rejected_requests_total
- apiserver_flowcontrol_dispatched_requests_total
- apiserver_flowcontrol_current_inqueue_requests
- apiserver_flowcontrol_current_executing_requests
Renamed PodHasNetwork to PodReadyToStartContainers.
Replaced apiserver_storage_db_total_size_in_bytes with apiserver_storage_size_bytes metric.
Scheduler now waits for handlers to finish syncing before the scheduling cycles start.
Set metrics-server's metric-resolution to 15s.
SubjectAccessReview requests sent to webhook authorizers now default spec.resourceAttributes.version to * if unset.
Supported specifying a custom retry period for cloud load-balancer operations.
The "value" part in the wait --for=jsonpath='{expression}'[=value] is now optional. If the value is not provided i.e., the command looks like wait --for=jsonpath='{expression}' then the wait condition is interpreted as matched when the expression returns any single JSON value like object or a literal.
The Kubernetes apiserver now emits a warning message for Pods with a null labelSelector in podAffinity or topologySpreadConstraints. The null labelSelector means "match none". Using it in podAffinity or topologySpreadConstraint could lead to unintended behavior.
The AdvancedAuditing feature gate that graduated to GA in v1.12 (and was unconditionally enabled) has been removed.
The ExpandedDNSConfig feature has graduated to GA. 'ExpandedDNSConfig' feature was locked to default value and will be removed in v1.30. If you were setting this feature gate explicitly, please remove it now.
The apiserver debug endpoint /debug/api_priority_and_fairness/dump_requests has been extended to dump executing requests as well as queued ones. A column for StartTime has been added to the returned table, with the queued requests having a StartTime of "0001-01-01T00:00:00Z". The executing requests have a RequestIndexInQueue of -1, and the QueueIndex is also -1 for priority levels without queues.
The helping message of commands which have sub-commands is now clearer and more instructive. It will show the full command instead of kubectl <command> --help ...

Changed kubectl create secret --help description. There will be a short introduction to the three secret types and clearer guidance on how to use the command.

The scheduler skips the InterPodAffinity Score plugin when nothing to do with the Pod. It will affect some metrics values related to the InterPodAffinity Score plugin.
The scheduler skips the PodTopologySpread Filter plugin if no spread constraints. It will affect some metrics values related to the PodTopologySpread Filter plugin.
The scheduler skips the PodTopologySpread Score plugin when nothing to do with the Pod. It will affect some metrics values related to the PodTopologySpread Score plugin.
The short names vwc and mwc were introduced for the resources validatingwebhookconfigurations and mutatingwebhookconfigurations.
Updated etcd image to 3.5.9-0.
Updated cAdvisor to v0.47.2 and fixed metrics in cri-o when a container restarts.
Updated distroless I-tables to use registry.k8s.io/build-image/distroless-iptables:v0.2.5
Updated distroless iptables to use released image registry.k8s.io/build-image/distroless-iptables:v0.2.4
Updated the scheduler interface and cache methods to use contextual logging.
ValidatingAdmissionPolicy type checking now correctly handles authorizer variable.
When a pod is done or not going to run, then ResourceClaims for it can be reused by other pods or deleted.
With the KubeletCgroupDriverFromCRI feature gate enabled and sufficiently new version of a container runtime, kubelet automatically detects the cgroup driver config from the container runtime, eliminating the need to specify the cgroupDriver configuration option (or --cgroup-driver flag) of kubelet.
[Kube-proxy]: Implemented connection draining for terminating nodes.
--version=v1.X.Y... can now be used to set the prerelease and buildID portions of the version reported by components.
RetroactiveDefaultStorageClass feature made stable and enabled by default.
TopologyManagerPolicyOptions feature-flag is promoted to beta and enabled by default.
force_delete_pods_total and force_delete_pod_errors_total metrics count all pod deletion behaviors.
klog text output now uses JSON as encoding for structs, maps and slices.
kube-proxy in iptables mode will now have separate sync_full_proxy_rules_duration_seconds\nand sync_partial_proxy_rules_duration_seconds (in addition to the existing\nsync_proxy_rules_duration_seconds), giving better information about the duration of each \nsync type, rather than only giving a weighted average of the two sync types together.
kubeadm: added a new "kubeadm config validate" command that can be used to validate any input config file. Use the --config flag to pass a config file to it. See the command --help screen for more information. As a result of adding this new command, enhance the validation capabilities of the existing "kubeadm config migrate" command. For both commands unknown APIs or fields will throw errors.
kubeadm: added the --allow-experimental-api flag to "kubeadm config migrate/validate" commands. It can be used to migrate or validate WIP/experimental APIs in the future.
kubeadm: generate CA certificates with a start time that is offset 5 minutes in the past relative to the current system time to workaround cases of clock desync.
plugin_evaluation_total metric supports prescore/score extension point. The metric doesn't get incremented when the prescore/score plugin has nothing to do with an incoming pod.

Logstash 8.9.1
Notable issues fixed

Fix pipeline to pipeline communication when upstream pipeline is terminated and events is written to a closed queue in downstream.
Fix DLQ unable to finalize segment error

Updates to dependencies

Update JDK to 17.0.8+7

Plugins
Elasticsearch Filter - 3.15.2

Added checking to ensure either query or query_template is non empty

Snmp Input - 1.3.3

Silence warnings when loading dictionary MIB files

Aws Integration - 7.1.5

Fix external documentation links

RabbitMQ 3.12.3
Core Server
Bug Fixes

Certain diagnostics operations during rolling upgrades from 3.10 to 3.11 could fail
if the listener_records_in_ets feature flag was enabled in the middle of the upgrade.
On Windows, PowerShell will no longer be used as a fallback for handle.exe
for computing how many file and socket handles a node uses.

If a user does not have handle.exe`` installed in the PATH`` of their Windows system,
a message will be logged once, and then the total handles being used will be set to 0.

PowerShell ended up being a CPU-intensive alternative that's not worth the gains
for many installations.

Node maintenance state was not replicated to all nodes, even though it was accessible
from any node (and for any node).

CLI Tools
Enhacements

Some warnings were emitted even when --formatter was set to json.

MQTT Plugin
Bug Fixes

MQTT connections could run into an exception when a queue it consumed from was temporarily
unavailable (e.g. was undergoing a leader election).

Enhancements

When QoS 0 consumers consistently do not keep up with publishers, some messages will be dropped
to avoid runaway resource usage.

Now the number of dropped messages will be reflected in the dropped message metric, together with unroutable messages.

HTTP AuthN/AuthZ Backend Plugin
Bug Fixes

AMQP 1.0 client connections were refused with this plugin.

LDAP AuthN/AuthZ Backend Plugin
Bug Fixes

AMQP 1.0 client connections were refused with this plugin.

Sonatype Nexus Repository 3.59.0
FIXES
NEXUS-39797: Resolved an issue that was causing some components to not be indexed for search in HA deployments.
NEXUS-39774 & 39573: Using the Search API to return Maven assets with an empty maven.classifier now works as expected.
NEXUS-39255: The Conan v2 remote list command to retrieve revisions performs as expected without a 500 error.
NEXUS-36486: The blobCreated date is now preserved when migrating to PostgreSQL.
NEXUS-36415: Adjusted handling in cases where invalid content violating metadata format is cached in a proxy repository.
NEXUS-35977: Improved error messaging and documentation related to requesting files from a R format repository.

Ansible AWX 22.7.0

Fix linting
Bump python-daemon package
Fix trial status and host limit with sub
Update example service-account.yml for container group in documentation
Add PR check to ensure JIRA links are present
Fix RBAC around credential access add button
Wait for new label IDs before setting label prompt values.
Add Request time out option for collection
Only process ansible_facts for successful jobs
Fix broken link to upgrade docs.
Allow importing licenses with a missing "usage" attribute
Remove extra quote from Skipped task status string
Modify main/0185 to set aside the json fields that might be a problem
Integrate scheduler into dispatcher main loop
HostMetricSummaryMonthly: Analytics export
Add a retry to update host facts on deadlocks
Fix programming error in facts retry merge
Make the default JOB_EVENT_BUFFER_SECONDS 1 seconds

Gitlab Community Edition 16.3.0
Added (169 changes)
Fixed (180 changes)
Changed (265 changes)
Security (22 changes)
*Use component to hide sensitive analytics settings (merge request)
*Fix undefined method page error in list dependencies (merge request)
*Fix undefined method licenses for nil:NilClass bug (merge request)
*Add pagination for license scanning (merge request)
*Mitigate autolink filter ReDOS (merge request)
*Revert 'security-408388--protected-branch' (merge request)
*Fix bug where comments on files with incorrect sha breaks UI (merge request)
*Prevent leaking emails of newly created users (merge request)
*Sanitize multiple hardlinks from import archives (merge request)
*Mitigate project reference filter ReDOS (merge request)
*Relocate PlantUML config and disable SVG support (merge request)
*Added redirect to filtered params (merge request)
*Validates project path availability (merge request)
*Fix XSS vector in Web IDE (merge request)
*Prevent creation of tags matching protected branch names (merge request)
*Add a stricter regex for the Harbor search param (merge request)
*Prohibit 40 character hex plus a hyphen if branch name is path (merge request)
*Fix policy project assign (merge request)
*Fix pipeline schedule authorization for protected branch/tag (merge request)
*Update pipeline user to the last policy MR author (merge request)
*Test nr 3: fast security->canonical sync (merge request)
*Test fast security->canonical sync (merge request)
Performance (17 changes)
Other (90 changes)

OpenUpdate - August 17, 2023

Stay Informed

This week, read about:

Finding the Best Linux Distro for Your Organization.
Oracle, SUSE, and others caught up in RHEL drama hit back with OpenELA.
What's New in Apache Kafka 3.5: Features to Watch.

Key Security, Maintenance, and Features Releases

Non-Security Based Updates

Angular 16.2.0
benchpress:
fix: correctly report GC memory amounts (#50760)
common:
feat: add component input binding support for NgComponentOutlet (#51148)
feat: Allow ngSrc to be changed post-init (#50683)
compiler:
feat: scope selectors in @scope queries (#50747)
compiler-cli:
fix: libraries compiled with v16.1+ breaking with Angular framework v16.0.x (#50714)
core:
feat: add afterRender and afterNextRender (#50607)
feat: create injector debugging APIs (#48639)
feat: support Provider type in Injector.create (#49587)
fix: handle hydration of view containers for root components (#51247)
router:
feat: exposes the fixture of the RouterTestingHarness (#50280)

Apache Tomcat 11.0.0-M10
Catalina:

Fix potential database connection leaks in DataSourceUserDatabase identified by Coverity Scan. (markt)
Make parsing of ExtendedAccessLogValve patterns more robust. (markt)
Fix failure trying to persist configuration for an internal credential handler. (remm)
66680: When serializing a session during the session presistence process, do not log a warning that null Principals are not serializable. Pull request #638 provided by tsryo. (markt)
66822: Use the same naming format in log messages for Connector instances as the associated ProtocolHandler instance. (markt)
The parts count should also lower the actual maxParameterCount used for parsing parameters if parts are parsed first. (remm)

Coyote

Refactor blocking reads and writes for the NIO connector to remove code paths that could allow a notification from the Poller to be missed resuting in a timeout rather than the expected read or write. (markt)
Refactor waiting for an HTTP/2 stream or connection window update to handle spurious wake-ups during the wait. (markt)
Improve extensibility of endpoints for socket channel creation and TLS. Pull request #639 provided by Marco Fargetta. (remm)
Correct a regression introduced in 11.0.0-M9 and use the correct constant when constructing the default value for the certificateKeystoreFile attribute of an SSLHostConfigCertificate instance. (markt)
Refactor HTTP/2 implementation to reduce pinning when using virtual threads. (markt)
Pass through ciphers referring to an OpenSSL profile, such a PROFILE=SYSTEM instead of producing an error trying to parse it. (remm)
66841: Ensure that AsyncListener.onError() is called after an error during asynchronous processing with HTTP/2. (markt)
66842: When using asynchronous I/O (the default), include DATA frames when calculating the HTTP/2 overhead count to ensure that connections are not prematurely terminated. (markt)
Correct a race condition that could cause spurious RST messages to be sent after the response had been written to an HTTP/2 stream. (markt)

Web-socket:

66681: Fix a NullPointerException when flushing batched messages with compression enabled using permessage-deflate. (markt)

Jdbc-pool:

Fix the releaseIdleCounter does not increment when testAllIdle releases them. Pull request #241 provided by Arun Chaitanya Miriappalli (lihan)
Fix the ConnectionState state will be inconsistent with actual state on the connection when an exception occurs while writing. Pull request #643 provided by Wenjun Xiao. (lihan)

Other:

Update NSIS to 3.0.9. (markt)
Update Checkstyle to 10.12.2. (markt)
Improvements to French translations. (remm)
Improvements to Japanese translations. Contributed by tak7iji and Shirayuking. (markt)
66829: Fix quoting so users can use the _RUNJAVA environment variable as intended on Windows when the path to the Java executable contains spaces. (markt)
66834: Correct the OSGi contract references in the manifest files to refer to the Jakarta EE contract names rather than the Java EE contract names. (markt)
Update Tomcat Native to 2.0.5. (markt)

Apache Tomcat 10.1.12
Catalina:

66680: When serializing a session during the session presistence process, do not log a warning that null Principals are not serializable. Pull request #638 provided by tsryo. (markt)
Catch NamingException in JNDIRealm#getPrincipal. It is used in Java up to 17 to signal closed connections. (fschumacher)
66822: Use the same naming format in log messages for Connector instances as the associated ProtocolHandler instance. (markt)
The parts count should also lower the actual maxParameterCount used for parsing parameters if parts are parsed first. (remm)

Coyote:

Correct a regression introduced in 10.1.11 and use the correct constant when constructing the default value for the certificateKeystoreFile attribute of an SSLHostConfigCertificate instance. (markt)
Refactor HTTP/2 implementation to reduce pinning when using virtual threads. (markt)
Pass through ciphers referring to an OpenSSL profile, such as PROFILE=SYSTEM instead of producing an error trying to parse it. (remm)
66841: Ensure that AsyncListener.onError() is called after an error during asynchronous processing with HTTP/2. (markt)
66842: When using asynchronous I/O (the default), include DATA frames when calculating the HTTP/2 overhead count to ensure that connections are not prematurely terminated. (markt)
Correct a race condition that could cause spurious RST messages to be sent after the response had been written to an HTTP/2 stream. (markt)

WebSocket:

66681: Fix a NullPointerException when flushing batched messages with compression enabled using permessage-deflate. (markt)

jdbc-pool:

Fix the releaseIdleCounter does not increment when testAllIdle releases them. Pull request #241 provided by Arun Chaitanya Miriappalli (lihan)
Fix the ConnectionState state will be inconsistent with actual state on the connection when an exception occurs while writing. Pull request #643 provided by Wenjun Xiao. (lihan)

Other:

Update NSIS to 3.0.9. (markt)
Update Checkstyle to 10.12.2. (markt)
Improvements to French translations. (remm)
Improvements to Japanese translations. Contributed by tak7iji and Shirayuking. (markt)
66829: Fix quoting so users can use the _RUNJAVA environment variable as intended on Windows when the path to the Java executable contains spaces. (markt)
66834: Correct the OSGi contract references in the manifest files to refer to the Jakarta EE contract names rather than the Java EE contract names. (markt)
Update Tomcat Native to 2.0.5. (markt)

Docker Engine / Compose v2.20.3
Enhancements:

Watch: add tar sync implementation by @milas in #10853
Improve buildkit node creation by @silvin-lubecki in #10843
Display builder's name on the first build line. by @silvin-lubecki in #10881
Improve shell completion for --project-directory by @relrelb in #10879
Add shell completion for --profile by @relrelb in #10878

Fixes:

Progress: minor correctness fixes by @milas in #10871
Up: do not warn on successful optional dependency complete by @milas in #10870
Build: fix missing proxy build args for classic builder by @milas in #10887

Internal:

Trace: do not block connecting to OTLP endpoint by @milas in #10882
Test: fix e2e test for privileged builds by @milas in #10873
Test: temporarily disable an exit-code-from Cucumber test case by @milas in #10875
Watch: support multiple containers for tar implementation by @milas in #10860
Watch: batch & de-duplicate file events by @milas in #10865
Watch: enable tar-based syncer by default by @milas in #10877
Update Config comment in API Service interface by @prafgup in #10840
Update README and CI workflows to match main branch by @glours in #10889

Dependencies:

Build(deps): bump github.com/docker/cli from 24.0.4+incompatible to 24.0.5+incompatible by @dependabot in #10845
Build(deps): bump google.golang.org/grpc from 1.56.2 to 1.57.0 by @dependabot in #10847
Build(deps): bump github.com/containerd/containerd from 1.7.2 to 1.7.3 by @dependabot in #10850
Build(deps): bump github.com/docker/docker from 24.0.5-0.20230714235725-36e9e796c6fc+incompatible to 24.0.5+incompatible by @dependabot in #10844
Update to go1.20.7 by @thaJeztah in #10861
Upgrade Golang to 1.21 by @glours in #10890
Bump compose-go to version v1.18.0 by @glours in #10891
Bump compose-go to version v1.18.1 by @glours in #10893
Build(deps): bump github.com/moby/buildkit from 0.12.1-0.20230717122532-faa0cc7da353 to 0.12.1 by @dependabot in #10867

HAProxy v2.9-dev3
BUG/MINOR: ssl: OCSP callback only registered for first SSL_CTX
BUG/MEDIUM: h3: Properly report a C-L header was found to the HTX start-line
MINOR: sample: add pid sample
MINOR: sample: implement act_conn sample fetch
MINOR: sample: accept_date / request_date return %Ts / %tr timestamp values
MEDIUM: sample: implement us and ms variant of utime and ltime
BUG/MINOR: sample: check alloc_trash_chunk() in conv_time_common()
DOC: configuration: describe Td in Timing events
MINOR: sample: implement the T* timer tags from the log-format as fetches
DOC: configuration: add sample fetches for timing events
BUG/MINOR: quic: Possible crash when acknowledging Initial v2 packets
MINOR: quic: Export QUIC traces code from quic_conn.c
MINOR: quic: Export QUIC CLI code from quic_conn.c
MINOR: quic: Move TLS related code to quic_tls.c
MINOR: quic: Add new "QUIC over SSL" C module.
MINOR: quic: Add a new quic_ack.c C module for QUIC acknowledgements
CLEANUP: quic: Defined but no more used function (quic_get_tls_enc_levels())
MINOR: quic: Split QUIC connection code into three parts
CLEANUP: quic: quic_conn struct cleanup
MINOR: quic; Move the QUIC frame pool to its proper location
BUG/MINOR: chunk: fix chunk_appendf() to not write a zero if buffer is full
BUG/MEDIUM: h3: Be sure to handle fin bit on the last DATA frame
DOC: configuration: rework the custom log format table
BUG/MINOR: quic+openssl_compat: Non initialized TLS encryption levels
CLEANUP: acl: remove cache_idx from acl struct
REORG: cfgparse: extract curproxy as a global variable
MINOR: acl: add acl() sample fetch
BUILD: cfgparse: keep a single "curproxy"
BUG/MEDIUM: bwlim: Reset analyse expiration date when then channel analyse ends
MEDIUM: stream: Reset response analyse expiration date if there is no analyzer
BUG/MINOR: htx/mux-h1: Properly handle bodyless responses when splicing is used
BUG/MEDIUM: quic: consume contig space on requeue datagram
BUG/MINOR: http-client: Don't forget to commit changes on HTX message
CLEANUP: stconn: Move comment about sedesc fields on the field line
REGTESTS: http: Create a dedicated script to test spliced bodyless responses
REGTESTS: Test SPLICE feature is enabled to execute script about splicing
BUG/MINOR: quic: reappend rxbuf buffer on fake dgram alloc error
BUILD: quic: fix wrong potential NULL dereference
MINOR: h3: abort request if not completed before full response
BUG/MAJOR: http-ana: Get a fresh trash buffer for each header value replacement
CLEANUP: quic: Remove quic_path_room().
MINOR: quic: Amplification limit handling sanitization.
MINOR: quic: Move some counters from [rt]x quic_conn anonymous struct
MEDIUM: quic: Send CONNECTION_CLOSE packets from a dedicated buffer.
MINOR: quic: Use a pool for the connection ID tree.
MEDIUM: quic: Allow the quic_conn memory to be asap released.
MINOR: quic: Release asap quic_conn memory (application level)
MINOR: quic: Release asap quic_conn memory from ->close() xprt callback.
MINOR: quic: Warning for OpenSSL wrapper QUIC bindings without "limited-quic"
REORG: http: move has_forbidden_char() from h2.c to http.h
BUG/MAJOR: h3: reject header values containing invalid chars
MINOR: mux-h2/traces: also suggest invalid header upon parsing error
MINOR: ist: add new function ist_find_range() to find a character range
MINOR: http: add new function http_path_has_forbidden_char()
MINOR: h2: pass accept-invalid-http-request down the request parser
REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests
BUG/MINOR: h1: do not accept '#' as part of the URI component
BUG/MINOR: h2: reject more chars from the :path pseudo header
BUG/MINOR: h3: reject more chars from the :path pseudo header
REGTESTS: http-rules: verify that we block '#' by default for normalize-uri
DOC: clarify the handling of URL fragments in requests
BUG/MAJOR: http: reject any empty content-length header value
BUG/MINOR: http: skip leading zeroes in content-length values
BUG/MEDIUM: mux-h1: fix incorrect state checking in h1_process_mux()
BUG/MEDIUM: mux-h1: do not forget EOH even when no header is sent
BUILD: mux-h1: shut a build warning on clang from previous commit
DEV: makefile: add a new "range" target to iteratively build all commits
CI: do not use "groupinstall" for Fedora Rawhide builds
CI: get rid of travis-ci wrapper for Coverity scan
BUG/MINOR: quic: mux started when releasing quic_conn
BUG/MINOR: quic: Possible crash in quic_cc_conn_io_cb() traces.
MINOR: quic: Add a trace for QUIC conn fd ready for receive
BUG/MINOR: quic: Possible crash when issuing "show fd/sess" CLI commands
BUG/MINOR: quic: Missing tasklet (quic_cc_conn_io_cb) memory release (leak)
BUG/MEDIUM: quic: fix tasklet_wakeup loop on connection closing
BUG/MINOR: hlua: fix invalid use of lua_pop on error paths
MINOR: hlua: add hlua_stream_ctx_prepare helper function
BUG/MEDIUM: hlua: streams don't support mixing lua-load with lua-load-per-thread
MAJOR: threads/plock: update the embedded library again
MINOR: stick-table: move the task_queue() call outside of the lock
MINOR: stick-table: move the task_wakeup() call outside of the lock
MEDIUM: stick-table: change the ref_cnt atomically
MINOR: stick-table: better organize the struct stktable
MEDIUM: peers: update ->commitupdate out of the lock using a CAS
MEDIUM: peers: drop then re-acquire the wrlock in peer_send_teachmsgs()
MEDIUM: peers: only read-lock peer_send_teachmsgs()
MEDIUM: stick-table: use a distinct lock for the updates tree
MEDIUM: stick-table: touch updates under an upgradable read lock
MEDIUM: peers: drop the stick-table lock before entering peer_send_teachmsgs()
MINOR: stick-table: move the update lock into its own cache line
CLEANUP: stick-table: slightly reorder the stktable struct
BUILD: defaults: use __WORDSIZE not LONGBITS for MAX_THREADS_PER_GROUP
MINOR: tools: make ptr_hash() support 0-bit outputs
MINOR: tools: improve ptr hash distribution on 64 bits
OPTIM: tools: improve hash distribution using a better prime seed
OPTIM: pools: use exponential back-off on shared pool allocation/release
OPTIM: pools: make pool_get_from_os() / pool_put_to_os() not update ->allocated
MINOR: pools: introduce the use of multiple buckets
MEDIUM: pools: spread the allocated counter over a few buckets
MEDIUM: pools: move the used counter over a few buckets
MEDIUM: pools: move the needed_avg counter over a few buckets
MINOR: pools: move the failed allocation counter over a few buckets
MAJOR: pools: move the shared pool's free_list over multiple buckets
MINOR: pools: make pool_evict_last_items() use pool_put_to_os_no_dec()
BUILD: pools: fix build error on clang with inline vs forceinline

Jenkins 2.419
Use standard size node icon even with long node names. (pull 8089)

Jenkins 2.418
New login page breaks login theme plugin. (issue 71238)
Fix "Manage Jenkins" context menu (regression in 2.415). (issue 71744)
Fix mistranslation of Japanese message in mailing list reference. (pull 8324)

Nodejs v20.5.1
Notable Changes
The following CVEs are fixed in this release:
CVE-2023-32002: Policies can be bypassed via Module._load (High)
CVE-2023-32558: process.binding() can bypass the permission model through path traversal (High)
CVE-2023-32004: Permission model can be bypassed by specifying a path traversal sequence in a Buffer (High)
CVE-2023-32006: Policies can be bypassed by module.constructor.createRequire (Medium)
CVE-2023-32559: Policies can be bypassed via process.binding (Medium)
CVE-2023-32005: fs.statfs can bypass the permission model (Low)
CVE-2023-32003: fs.mkdtemp() and fs.mkdtempSync() can bypass the permission model (Low)

OpenSSL Security Releases:
OpenSSL security advisory 14th July.
OpenSSL security advisory 19th July.
OpenSSL security advisory 31st July.

Postgres REL_15_4

Disallow substituting a schema or owner name into an extension script if the name contains a quote, backslash, or dollar sign (Noah Misch)

This restriction guards against SQL-injection hazards for trusted extensions.

The PostgreSQL Project thanks Micah Gate, Valerie Woolard, Tim Carey-Smith, and Christoph Berg for reporting this problem. (CVE-2023-39417)

Fix MERGE to enforce row security policies properly (Dean Rasheed)

When MERGE performs an UPDATE action, it should enforce any UPDATE or SELECT RLS policies defined on the target table, to be consistent with the way that a plain UPDATE with a WHERE clause works. Instead it was enforcing INSERT RLS policies for both INSERT and UPDATE actions.

In addition, when MERGE performs a DO NOTHING action, it applied the target table's DELETE RLS policies to existing rows, even though those rows are not being deleted. While it's not a security problem, this could result in unwanted errors.

The PostgreSQL Project thanks Dean Rasheed for reporting this problem. (CVE-2023-39418)

Fix confusion between empty (no rows) ranges and all-NULL ranges in BRIN indexes, as well as incorrect merging of all-NULL summaries (Tomas Vondra)

Each of these oversights could result in forgetting that a BRIN index range contains any NULL values, potentially allowing subsequent queries that should return NULL values to miss doing so.

This fix will not in itself correct faulty BRIN entries. It's recommended to REINDEX any BRIN indexes that may be used to search for nulls.

Avoid leaving a corrupted database behind when DROP DATABASE is interrupted (Andres Freund)

If DROP DATABASE was interrupted after it had already begun taking irreversible steps, the target database remained accessible (because the removal of its pg_database row would roll back), but it would have corrupt contents. Fix by marking the database as inaccessible before we begin to perform irreversible operations. A failure after that will leave the database still partially present, but nothing can be done with it except to issue another DROP DATABASE.

Ensure that partitioned indexes are correctly marked as valid or not at creation (Michael Paquier)

If a new partitioned index matches an existing but invalid index on one of the partitions, the partitioned index could end up being marked valid prematurely. This could lead to misbehavior or assertion failures in subsequent queries on the partitioned table.

Ignore invalid child indexes when matching partitioned indexes to child indexes during ALTER TABLE ATTACH PARTITION (Michael Paquier)

Such an index will now be ignored, and a new child index created instead.

Fix possible failure when marking a partitioned index valid after all of its partitions have been attached (Michael Paquier)

The update of the index's pg_index entry could use stale data for other columns. One reported symptom is an “attempted to update invisible tuple” error.

Fix ALTER EXTENSION SET SCHEMA to complain if the extension contains any objects outside the extension's schema (Michael Paquier, Heikki Linnakangas)

Erroring out if the extension contains objects in multiple schemas was always intended; but the check was mis-coded so that it would fail to detect some cases, leading to surprising behavior.

Fix tracking of tables' access method dependencies (Michael Paquier)

ALTER TABLE ... SET ACCESS METHOD failed to update relevant pg_depend entries when changing a table's access method. When using non-built-in access methods, this creates a risk that an access method could be dropped even though tables still depend on it. This fix corrects the logic in ALTER TABLE, but it will not adjust any already-missing pg_depend entries.

Don't use partial unique indexes for uniqueness proofs in the planner (David Rowley)

This could give rise to incorrect plans, since the presumed uniqueness of rows read from a table might not hold if the index in question isn't used to scan the table.

Don't Memoize lateral joins with volatile join conditions (Richard Guo)

Applying Memoize to a sub-plan that contains volatile filter conditions is likely to lead to wrong answers. The check to avoid doing this missed some cases that can arise when using LATERAL.

Avoid producing incorrect plans for foreign joins with pseudoconstant join clauses (Etsuro Fujita)

The planner currently lacks support for attaching pseudoconstant join clauses to a pushed-down remote join, so disable generation of remote joins in such cases. (A better solution will require ABI-breaking changes of planner data structures, so it will have to wait for a future major release.)

Correctly handle sub-SELECTs in RLS policy expressions and security-barrier views when expanding rule actions (Tom Lane)
Fix race conditions in conflict detection for SERIALIZABLE isolation mode (Thomas Munro)

Conflicts could be missed when using bitmap heap scans, when using GIN indexes, and when examining an initially-empty btree index. All these cases could lead to serializability failures due to improperly allowing conflicting transactions to commit.

Fix misbehavior of EvalPlanQual checks with inherited or partitioned target tables (Tom Lane)

This oversight could lead to update or delete actions in READ COMMITTED isolation mode getting performed when they should have been skipped because of a conflicting concurrent update.

Fix hash join with an inner-side hash key that contains Params coming from an outer nested loop (Tom Lane)

When rescanning the join after the values of such Params have changed, we must rebuild the hash table, but neglected to do so. This could result in missing join output rows.

Fix intermittent failures when trying to update a field of a composite column (Tom Lane)

If the overall value of the composite column is wide enough to require out-of-line toasting, then an unluckily-timed cache flush could cause errors or server crashes.

Prevent query-lifespan memory leaks in some UPDATE queries with triggers (Tomas Vondra)
Prevent query-lifespan memory leaks when an Incremental Sort plan node is rescanned (James Coleman, Laurenz Albe, Tom Lane)
Accept fractional seconds in the input to jsonpath's datetime() method (Tom Lane)
Prevent stack-overflow crashes with very complex text search patterns (Tom Lane)
Allow tokens up to 10240 bytes long in pg_hba.conf and pg_ident.conf (Tom Lane)

The previous limit of 256 bytes has been found insufficient for some use-cases.

Ensure that all existing placeholders are checked for matches when an extension declares its GUC prefix to be reserved (Karina Litskevich, Ekaterina Sokolova)

Faulty loop logic could cause some entries to be skipped.

Fix mishandling of C++ out-of-memory conditions (Heikki Linnakangas)

If JIT is in use, running out of memory in a C++ new call would lead to a PostgreSQL FATAL error, instead of the expected C++ exception.

Fix rare null-pointer crash in plancache.c (Tom Lane)
Avoid leaking a stats entry for a subscription when it is dropped (Masahiko Sawada)
Avoid losing track of possibly-useful shared memory segments when a page free results in coalescing ranges of free space (Dongming Liu)

Ensure that the segment is moved into the appropriate “bin” for its new amount of free space, so that it will be found by subsequent searches.

Allow VACUUM to continue after detecting certain types of b-tree index corruption (Peter Geoghegan)

If an invalid sibling-page link is detected, log the issue and press on, rather than throwing an error as before. Nothing short of REINDEX will fix the broken index, but preventing VACUUM from completing until that is done risks making matters far worse.

Ensure that WrapLimitsVacuumLock is released after VACUUM detects invalid data in pg_database.datfrozenxid or pg_database.datminmxid (Andres Freund)

Failure to release this lock could lead to a deadlock later, although the lock would be cleaned up if the session exits or encounters some other error.

Avoid double replay of prepared transactions during crash recovery (suyu.cmj, Michael Paquier)

After a crash partway through a checkpoint with some two-phase transaction state data already flushed to disk by this checkpoint, crash recovery could attempt to replay the prepared transaction(s) twice, leading to a fatal error such as “lock is already held” in the startup process.

Ensure that a newly created, but still empty table is fsync'ed at the next checkpoint (Heikki Linnakangas)

Without this, if there is an operating system crash causing the empty file to disappear, subsequent operations on the table might fail with “could not open file” errors.

Ensure that creation of the init fork of an unlogged index is WAL-logged (Heikki Linnakangas)

While an unlogged index's main data fork is not WAL-logged, its init fork should be, to ensure that we have a consistent state to restore the index to after a crash. This step was missed if the init fork contains no data, which is a case not used by any standard index AM; but perhaps some extension behaves that way.

Silence bogus “missing contrecord” errors (Thomas Munro)

Treat this case as plain end-of-WAL to avoid logging inaccurate complaints from pg_waldump and walsender.

Fix overly strict assertion in jsonpath code (David Rowley)

This assertion failed if a query applied the .type() operator to a like_regex result. There was no bug in non-assert builds.

Avoid assertion failure when processing an empty statement via the extended query protocol in an already-aborted transaction (Tom Lane)
Avoid assertion failure when the stats_fetch_consistency setting is changed intra-transaction (Kyotaro Horiguchi)
Fix contrib/fuzzystrmatch's Soundex difference() function to handle empty input sanely (Alexander Lakhin, Tom Lane)

An input string containing no alphabetic characters resulted in unpredictable output.

Tighten whitespace checks in contrib/hstore input (Evan Jones)

In some cases, characters would be falsely recognized as whitespace and hence discarded.

Disallow oversize input arrays with contrib/intarray's gist__int_ops index opclass (Ankit Kumar Pandey, Alexander Lakhin)

Previously this code would report a NOTICE but press on anyway, creating an invalid index entry that presents a risk of crashes when the index is read.

Avoid useless double decompression of GiST index entries in contrib/intarray (Konstantin Knizhnik, Matthias van de Meent, Tom Lane)
Fix contrib/pageinspect's gist_page_items() function to work when there are included index columns (Alexander Lakhin, Michael Paquier)

Previously, if the index has included columns, gist_page_items() would fail to display those values on index leaf pages, or crash outright on non-leaf pages.

In psql, ignore the PSQL_WATCH_PAGER environment variable when stdin/stdout are not a terminal (Tom Lane)

This corresponds to the treatment of PSQL_PAGER in commands besides \watch.

Fix pg_dump to correctly handle new-style SQL-language functions whose bodies require parse-time dependencies on unique indexes (Tom Lane)

Such cases can arise from GROUP BY and ON CONFLICT clauses, for example. The function must then be postponed until after the unique index in the dump output, but pg_dump did not do that and instead printed a warning about “could not resolve dependency loop”.

Improve pg_dump's display of details about dependency-loop problems (Tom Lane)
Avoid crash in pgbench with an empty pipeline and prepared mode (Álvaro Herrera)
Ensure that pg_index.indisreplident is kept up-to-date in relation cache entries (Shruthi Gowda)

This value could be stale in some cases. There is no core code that relies on the relation cache's copy, so this is only a latent bug as far as Postgres itself is concerned; but there may be extensions for which it is a live bug.

Fix make_etags script to work with non-Exuberant ctags (Masahiko Sawada)

Postgres REL_14_9

Disallow substituting a schema or owner name into an extension script if the name contains a quote, backslash, or dollar sign (Noah Misch)

This restriction guards against SQL-injection hazards for trusted extensions.

The PostgreSQL Project thanks Micah Gate, Valerie Woolard, Tim Carey-Smith, and Christoph Berg for reporting this problem. (CVE-2023-39417)

Fix confusion between empty (no rows) ranges and all-NULL ranges in BRIN indexes, as well as incorrect merging of all-NULL summaries (Tomas Vondra)

Each of these oversights could result in forgetting that a BRIN index range contains any NULL values, potentially allowing subsequent queries that should return NULL values to miss doing so.

This fix will not in itself correct faulty BRIN entries. It's recommended to REINDEX any BRIN indexes that may be used to search for nulls.

Avoid leaving a corrupted database behind when DROP DATABASE is interrupted (Andres Freund)

Ensure that partitioned indexes are correctly marked as valid or not at creation (Michael Paquier)

Ignore invalid child indexes when matching partitioned indexes to child indexes during ALTER TABLE ATTACH PARTITION (Michael Paquier)

Such an index will now be ignored, and a new child index created instead.

Fix possible failure when marking a partitioned index valid after all of its partitions have been attached (Michael Paquier)

The update of the index's pg_index entry could use stale data for other columns. One reported symptom is an “attempted to update invisible tuple” error.

Fix ALTER EXTENSION SET SCHEMA to complain if the extension contains any objects outside the extension's schema (Michael Paquier, Heikki Linnakangas)

Erroring out if the extension contains objects in multiple schemas was always intended; but the check was mis-coded so that it would fail to detect some cases, leading to surprising behavior.

Don't use partial unique indexes for uniqueness proofs in the planner (David Rowley)

This could give rise to incorrect plans, since the presumed uniqueness of rows read from a table might not hold if the index in question isn't used to scan the table.

Don't Memoize lateral joins with volatile join conditions (Richard Guo)

Applying Memoize to a sub-plan that contains volatile filter conditions is likely to lead to wrong answers. The check to avoid doing this missed some cases that can arise when using LATERAL.

Avoid producing incorrect plans for foreign joins with pseudoconstant join clauses (Etsuro Fujita)

Correctly handle sub-SELECTs in RLS policy expressions and security-barrier views when expanding rule actions (Tom Lane)
Fix race conditions in conflict detection for SERIALIZABLE isolation mode (Thomas Munro)

Fix misbehavior of EvalPlanQual checks with inherited or partitioned target tables (Tom Lane)

This oversight could lead to update or delete actions in READ COMMITTED isolation mode getting performed when they should have been skipped because of a conflicting concurrent update.

Fix hash join with an inner-side hash key that contains Params coming from an outer nested loop (Tom Lane)

When rescanning the join after the values of such Params have changed, we must rebuild the hash table, but neglected to do so. This could result in missing join output rows.

Fix intermittent failures when trying to update a field of a composite column (Tom Lane)

If the overall value of the composite column is wide enough to require out-of-line toasting, then an unluckily-timed cache flush could cause errors or server crashes.

Prevent query-lifespan memory leaks in some UPDATE queries with triggers (Tomas Vondra)
Prevent query-lifespan memory leaks when an Incremental Sort plan node is rescanned (James Coleman, Laurenz Albe, Tom Lane)
Accept fractional seconds in the input to jsonpath's datetime() method (Tom Lane)
Prevent stack-overflow crashes with very complex text search patterns (Tom Lane)
Allow tokens up to 10240 bytes long in pg_hba.conf and pg_ident.conf (Tom Lane)

The previous limit of 256 bytes has been found insufficient for some use-cases.

Fix mishandling of C++ out-of-memory conditions (Heikki Linnakangas)

If JIT is in use, running out of memory in a C++ new call would lead to a PostgreSQL FATAL error, instead of the expected C++ exception.

Fix rare null-pointer crash in plancache.c (Tom Lane)
Avoid losing track of possibly-useful shared memory segments when a page free results in coalescing ranges of free space (Dongming Liu)

Ensure that the segment is moved into the appropriate “bin” for its new amount of free space, so that it will be found by subsequent searches.

Allow VACUUM to continue after detecting certain types of b-tree index corruption (Peter Geoghegan)

Ensure that WrapLimitsVacuumLock is released after VACUUM detects invalid data in pg_database.datfrozenxid or pg_database.datminmxid (Andres Freund)

Failure to release this lock could lead to a deadlock later, although the lock would be cleaned up if the session exits or encounters some other error.

Avoid double replay of prepared transactions during crash recovery (suyu.cmj, Michael Paquier)

Fix possible failure while promoting a standby server, if archiving is enabled and two-phase transactions need to be recovered (Julian Markwort)

If any required two-phase transactions were logged in the most recent (partial) log segment, promotion would fail with an incorrect complaint about “requested WAL segment has already been removed”.

Ensure that a newly created, but still empty table is fsync'ed at the next checkpoint (Heikki Linnakangas)

Without this, if there is an operating system crash causing the empty file to disappear, subsequent operations on the table might fail with “could not open file” errors.

Ensure that creation of the init fork of an unlogged index is WAL-logged (Heikki Linnakangas)

Fix missing reinitializations of delay-checkpoint-end flags (suyu.cmj)

This could result in unnecessary delays of checkpoints, or in assertion failures in assert-enabled builds.

Fix overly strict assertion in jsonpath code (David Rowley)

This assertion failed if a query applied the .type() operator to a like_regex result. There was no bug in non-assert builds.

Avoid assertion failure when processing an empty statement via the extended query protocol in an already-aborted transaction (Tom Lane)
Fix contrib/fuzzystrmatch's Soundex difference() function to handle empty input sanely (Alexander Lakhin, Tom Lane)

An input string containing no alphabetic characters resulted in unpredictable output.

Tighten whitespace checks in contrib/hstore input (Evan Jones)

In some cases, characters would be falsely recognized as whitespace and hence discarded.

Disallow oversize input arrays with contrib/intarray's gist__int_ops index opclass (Ankit Kumar Pandey, Alexander Lakhin)

Previously this code would report a NOTICE but press on anyway, creating an invalid index entry that presents a risk of crashes when the index is read.

Avoid useless double decompression of GiST index entries in contrib/intarray (Konstantin Knizhnik, Matthias van de Meent, Tom Lane)
Fix contrib/pageinspect's gist_page_items() function to work when there are included index columns (Alexander Lakhin, Michael Paquier)

Previously, if the index has included columns, gist_page_items() would fail to display those values on index leaf pages, or crash outright on non-leaf pages.

Fix pg_dump to correctly handle new-style SQL-language functions whose bodies require parse-time dependencies on unique indexes (Tom Lane)

Ensure that pg_index.indisreplident is kept up-to-date in relation cache entries (Shruthi Gowda)

Redis 7.2.0
Upgrade urgency LOW: This is the first stable Release for Redis 7.2.
Bug Fixes:

redis-cli in cluster mode handles unknown-endpoint (#12273)
Update request / response policy hints for a few commands (#12417)
Ensure that the function load timeout is disabled during loading from RDB/AOF and on replicas. (#12451)
Fix false success and a memory leak for ACL selector with bad parenthesis combination (#12452)
Fix the assertion when script timeout occurs after it signaled a blocked client (#12459)

Fixes for issues in previous releases of Redis 7.2:

Update MONITOR client's memory correctly for INFO and client-eviction (#12420)
The response of cluster nodes was unnecessarily adding an extra comma when no
hostname was present. (#12411)

OpenJ9 0.40.0

d12d10c (0.40) Update to OpenSSL 1.1.1v Peter Shipton #17896
67512b5 (0.40) Update OpenSSL to the 1.1.1 July 19 CVE level Peter Shipton #17836
18fb6d1 (0.40) Use jdk19 to build jdk20 Peter Shipton #17834
b681a67 (0.40) Exclude cmdLineTester_CryptoTest in FIPS mode Paritosh Kumar #17777
ac8c50c (v0.40.0-release) j9gc_createJavaLangString protects string objects across GC points Jason Feng #17747
7319b8d (0.40) Split sanity.openjdk into 3 parallel jobs Lan Xia #17705
6eed053 (v0.40.0-release) CRIU tests pass if the original thread IDs can't be acquired Jason Feng #17702
c5b1658 (0.40) Modify the translated PII files in nls folder 20230627 Dong Chen #17687
26d65ac Change API used for computing code cache size in low memory environments (0.40.0) Marius Pirvu #17682
4dd1080 (v0.40.0-release) CRIU tests require only one Pre-checkpoint message Jason Feng #17669
e116b33 (v0.40.0-release) CRIU skips clearInetAddressCache() if InetAddress is not initialized Jason Feng #17670
e13741a (0.40) Add missed check for compressed string Dmitri Pivkine #17661
558f239 (0.40) CRIU GC: Flush and Reset Buffers on Reinit Salman Rana #17653
c50c466 (0.40) Add checkpoint delay when clinit is occuring Tobi Ajila #17652
8b4420c (v0.40.0)Use debug interpreter unconditionally when debug is enabled … Mike Zhang #17627
efe6ee2 (v0.40.0-release) CRIU throws JVMCRIUException in single threaded mode if parks no timeout Jason Feng #17639
2684cbb (0.40) Update Split List Forced Flag + Revert CRIU Thread Count Reinit Salman Rana #17644
970c9be (0.40) GC CRIU: Reinit HeapRegionDescriptorExtensions (Region Obj Lists) Salman Rana #17645
71eab61 (0.40) Avoid generating store of uninitialized auto when reducing TRT2 Devin Papineau #17605
b5af32b [0.40] Add NLS message: J9NLS_PORT_RUNNING_IN_CONTAINER_FAILURE Babneet Singh #17600
17f2765 (0.40) Fix invalid OMR_PRI* usage on Windows Kevin Grigorenko #17569
c4720f2 [FFI/Jtreg_JDK20] Keep the downcall address alive for downcall (0.40) ChengJin01 #17565
936ec54 (0.40) Modify the translated PII files in nls folder 20230607 Dong Chen #17545
99c5d95 [FFI/Jtreg_JDK20] Validate the downcall address with the scope check (0.40) ChengJin01 #17538
b9cd65e Insert branch around re performing store for awrtbar Rahil Shah #17517
f514560 CRIU skips j9sysinfo_get_username()/getpwuid() if isCheckPointAllowed Jason Feng #17505
0a07503 Put select system property names and values in allocated memory Keith W. Campbell #17407
bedafef Handle new vector opcodes Gita Koblents #17112
60798a3 Revert "Enable EDO during AOT compilation" Peter Shipton #17512
6ed80ce Enable EDO during AOT compilation Christian Despres #17217
3cbf8a0 Bump actions/setup-python from 2.3.3 to 4.6.1 dependabot[bot] #17502
4334ef0 Remove configuration information for Java 19 Keith W. Campbell #17507
42d8c31 Correct return type of JVM_Sleep() Keith W. Campbell #17504
c005819 Expand bytecode offset variables to 32bit Kevin Langman #17469
91c8570 Fix array constructor for Object Lists Aleksandar Micic #17503
283b706 Set LIGHT_WEIGHT_CHECKOUT to true Lan Xia #17497
423823f Correct SPDX license identifiers Jason Feng #17494
b087017 Correct SPDX license identifier Dmitri Pivkine #17489
dd16eba CRIU JDK11UpTimeoutAdjustmentTest adjusts for thread starting Jason Feng #17473
9797bca Rework RegionExtenstion/Object List Initialization Salman Rana #17461
69d50bc Bump actions/github-script from 3.2.0 to 6.4.1 dependabot[bot] #17481
ba2ccc1 Bump actions/checkout from 2.7.0 to 3.5.2 dependabot[bot] #17482
914adf4 Bump adoptium/run-aqa from 1.0.8 to 2.0.1 dependabot[bot] #17483
54a776f Bump peter-evans/create-pull-request from 3.14.0 to 5.0.1 dependabot[bot] #17480
013e44d Bump actions/upload-artifact from 2.3.1 to 3.1.2 dependabot[bot] #17484
ff98e55 [StepSecurity] Apply security best practices StepSecurity Bot #17477
b58a15e Call static method VM.getVMArgs() from JNI as a static method Peter Shipton #17475
f98cb31 Update openssl to version 1.1.1u Keith W. Campbell #17468
3e340db Disable FFI specific code for compilation in JDK21 ChengJin01 #17352
6aab183 Add/update java.specification.maintenance.version Keith W. Campbell #17470
c7ac2f7 Correct SPDX license identifiers Keith W. Campbell #17435
3b029b0 Add support for persistent SCC on z/OS Hang Shao #17073
f988e15 Set symbol declared class for field shadows Devin Papineau #17327
623c7ba Adding helper functions for crc32 special routines to enable optimizations in AOT Bhavani SN #17453
1b94cba Handle code cache alloction for low memory SajinaKandy #17425
12286f5 CRIU restore clears InetAddress.cache Jason Feng #17448
ec0eb13 Add the unimplemented assertion to Thread.findScopedValueBindings() Gengchen Tuo #17451
3bacb5a Add CH Table AOT Feature Flag Irwin D'Souza #17260
4bb727b Place fatal asserts in FE queries that JITServer should not call Marius Pirvu #17355
18f6869 Simplify callMustBeInlinedRegardlessOfSize calls James You #17406
1220e36 Use genLoadProfiledClassAddressConstant in Z codegen Spencer Comin #14932
37e239e Revert "Sync JVM init and exit paths" Babneet Singh #17438
de38712 Fix bug related to J9::Options::_compilationDelayTime unit Marius Pirvu #17436
3ca50dc Fix compile error due to unused variable Keith W. Campbell #17434
7d5d62a Correctly handle primitive VTs in System.arraycopy Ehren Julien-Neitzert #17048
3fbe09e Add areFlattenableValueTypesEnabled() for JIT Hang Shao #17413
22b17b8 [Jtreg/FFI] Remove the null segment check for pointer ChengJin01 #17408
826d49a (0.39) Prototype Continuation caching Babneet Singh #17409
e4a741f Add new optimization catchBlockProfiler Marius Pirvu #16854
b182f7a Add 31-64 interop support for JVM_ funcs for JDK17+ Joran Siu #17369
9667d83 Add new build flag to split value object feature from Valhalla Hang Shao #17394
a555ad2 WIP: Teach ValueTypeUnsafeTests about dual header shape Shubham Verma #17375
cb36d2d Sync JVM init and exit paths Babneet Singh #17101
d41eba9 Fix handling of IPv6 addressed Keith W. Campbell #17403
c9ea68f Enable CRC32 to run with AOT enabled on Power Bhavani SN #17243
8800e58 Patch addresses in LLILF/IIHF pairs on class unload and HCR Spencer Comin #15705
0ef06f4 Use TRUE instead of true calling freeContinuation() Babneet Singh #17398
8aa8676 Prototype Continuation caching Jack Lu #17344
09a3602 DDR: Fix function call parsing in StackMap Devin Nakamura #17278
b5c39bf Return false from JVM_DTraceIsSupported Peter Shipton #17391
1cbe6d1 Add missing value type check before zero the lockword Hang Shao #17381
8e3bb68 [FFI/JDK20_Jtreg] Handle the invalid arguments & return value ChengJin01 #17308
7806354 Provide a better error message for failed library loads on jdk17+ Peter Shipton #17374
3e7e8f9 Fix to handle suspend/resume of virtual/carrier threads Dipak Bagadiya #17350
45ed10a Fix typo in JDK11 build instructions James You #17373
575cae3 Remove unnecessary compatibility constant J9DescriptionCpTypeShift Keith W. Campbell #17376
ae2bda7 Throw UnsupportedOperationException in sun.misc.Perf.attach natives Peter Shipton #17380
ba48d1f Refactor GC Object List Allocation/Initialization Salman Rana #17330
7aa3fb8 Introduce GC CRIU (reinit) API for Thread Local Obj Buffers / Env Delegate Salman Rana #17348
0d24025 Remove obsolete references to freetype in jdk8 build instructions Peter Shipton #17379
9f19595 Define J9ClassEnv::primitiveArrayComponentType() Devin Papineau #17274
400ef3e Fix constant mapping in J9ConstantPoolCommand Jack Lu #17371
61cabd5 Support offloading for jdk17+ Peter Shipton #17306
899eedf Ensure JITServer tests check if server exists Irwin D'Souza #17363
663c581 Correct condition for preparing offloading library Keith W. Campbell #17370
7815549 Close VM and thread libraries on successful DestroyJavaVM Graham Chapman #17336
b3ac5be Correct types for min, length in memory segment objects Keith W. Campbell #17275
e698b8f Revert "Restore @OverRide annotation for Access.getLoaderNameID()" Peter Shipton #17361
36f6357 Implement JVM_VirtualThreadHideFrames() Gengchen Tuo #16654

OpenUpdate - August 10, 2023

Stay Informed

This week, read about:

New P2PInfect Worm Targeting Redis Servers on Linux and Windows Systems.
New Acoustic Attack Steals Data From Keystrokes With 95% Accuracy.
Unpacking Open Source Compliance.
We're in the OWASP-Makes-List-of-Security-Bug-Types Phase with LLM Chatbots.

Key Security, Maintenance, and Features Releases

Security Based Updates

Updates to the OpenLogic CentOS Repository
OpenLogic’s Enterprise Linux Team has recently published the following updates:

CVE-2022-29154
CentOS 8
rsync-3.1.3-12_ol001

We recommend that you update your CentOS 8 systems to protect against this vulnerability. As usual, please ensure that you test these updates before deploying to production.

If you don't currently have CentOS repo access, please reach out to your Perforce/OpenLogic salesperson … you may already be entitled to access with your existing support contract!

Zookeeper 3.9.0
Bugs:
ZOOKEEPER-2108 - Compilation error in ZkAdaptor.cc with GCC 4.7 or later
ZOOKEEPER-3652 - Improper synchronization in ClientCnxn
ZOOKEEPER-3908 - zktreeutil multiple issues
ZOOKEEPER-3996 - Flaky test: ReadOnlyModeTest.testConnectionEvents
ZOOKEEPER-4026 - CREATE2 requests embeded in a MULTI request only get a regular CREATE response
ZOOKEEPER-4296 - NullPointerException when ClientCnxnSocketNetty is closed without being opened
ZOOKEEPER-4308 - Flaky test: EagerACLFilterTest.testSetDataFail
ZOOKEEPER-4393 - Problem to connect to zookeeper in FIPS mode
ZOOKEEPER-4466 - Support different watch modes on same path
ZOOKEEPER-4471 - Remove WatcherType.Children break persistent watcher's child events
ZOOKEEPER-4473 - zooInspector create root node fail with path validate
ZOOKEEPER-4475 - Persistent recursive watcher got NodeChildrenChanged event
ZOOKEEPER-4477 - Single Kerberos ticket renewal failure can prevent all future renewals since Java 9
ZOOKEEPER-4504 - ZKUtil#deleteRecursive causing deadlock in HDFS HA functionality
ZOOKEEPER-4505 - CVE-2020-36518 - Upgrade jackson databind to 2.13.2.1
ZOOKEEPER-4511 - Flaky test: FileTxnSnapLogMetricsTest.testFileTxnSnapLogMetrics
ZOOKEEPER-4514 - ClientCnxnSocketNetty throwing NPE
ZOOKEEPER-4515 - ZK Cli quit command always logs error
ZOOKEEPER-4537 - Race between SyncThread and CommitProcessor thread
ZOOKEEPER-4549 - ProviderRegistry may be repeatedly initialized
ZOOKEEPER-4565 - Config watch path get truncated abnormally and fail chroot zookeeper client
ZOOKEEPER-4647 - Tests don't pass on JDK20 because we try to mock InetAddress
ZOOKEEPER-4654 - Fix C client test compilation error in Util.cc.
ZOOKEEPER-4674 - C client tests don't pass on CI
ZOOKEEPER-4719 - Use bouncycastle jdk18on instead of jdk15on
ZOOKEEPER-4721 - Upgrade OWASP Dependency Check to 8.3.1

New Features:
ZOOKEEPER-4570 - Admin server API for taking snapshot and stream out the data
ZOOKEEPER-4655 - Communicate the Zxid that triggered a WatchEvent to fire

Improvements:
ZOOKEEPER-3731 - Disable HTTP TRACE Method
ZOOKEEPER-3806 - TLS - dynamic loading for client trust/key store
ZOOKEEPER-3860 - Avoid reverse DNS lookup for hostname verification when hostnames are provided in the connection url
ZOOKEEPER-4289 - Reduce the performance impact of Prometheus metrics
ZOOKEEPER-4303 - ZooKeeperServerEmbedded could auto-assign and expose ports
ZOOKEEPER-4464 - zooinspector display "Ephemeral Owner" in hex for easy match to jmx session
ZOOKEEPER-4467 - Missing op code (addWatch) in Request.op2String
ZOOKEEPER-4472 - Support persistent watchers removing individually
ZOOKEEPER-4474 - ZooDefs.opNames is unused
ZOOKEEPER-4490 - Publish Clover results to SonarQube
ZOOKEEPER-4491 - Adding SSL support to Zktreeutil
ZOOKEEPER-4492 - Merge readOnly field into ConnectRequest and Response
ZOOKEEPER-4494 - Fix error message format
ZOOKEEPER-4518 - remove useless log in the PrepRequestProcessor#pRequest method
ZOOKEEPER-4519 - Testable interface should have a testableCloseSocket() method
ZOOKEEPER-4529 - Upgrade netty to 4.1.76.Final
ZOOKEEPER-4531 - Revert Netty TCNative change
ZOOKEEPER-4551 - Do not log spammy stacktrace when a client closes its connection
ZOOKEEPER-4566 - Create tool for recursive snapshot analysis
ZOOKEEPER-4573 - Encapsulate request bytebuffer in Request
ZOOKEEPER-4575 - ZooKeeperServer#processPacket take record instead of bytes
ZOOKEEPER-4616 - Upgrade docker image for the dev enviroment to resolve CVEs
ZOOKEEPER-4622 - Add Netty-TcNative OpenSSL Support
ZOOKEEPER-4636 - Fix zkServer.sh for AIX
ZOOKEEPER-4657 - Publish SBOM artifacts
ZOOKEEPER-4659 - Upgrade Commons CLI to 1.5.0 due to OWASP failing on 1.4 CVE-2021-37533
ZOOKEEPER-4660 - Suppress false positive OWASP failure for CVE-2021-37533
ZOOKEEPER-4661 - Upgrade Jackson Databind to 2.13.4.2 for CVE-2022-42003 CVE-2022-42004
ZOOKEEPER-4705 - Restrict GitHub merge button to allow squash commit only
ZOOKEEPER-4717 - Cache serialize data in the request to avoid repeat serialize.
ZOOKEEPER-4718 - Removing unnecessary heap memory allocation in serialization can help reduce GC pressure.

Gitlab Community 16.2.2
Added (1 change):
Add MR reviewers to BitBucketServer import to 16-2

Fixed (2 changes):
Disable IAT verification by default
Enable descendant_security_scans by default GitLab Enterprise Edition

Security (17 changes):
Fix undefined method licenses for nil:NilClass bug (merge request)
Fix undefined method page error in list dependencies (merge request)
Add pagination for license scanning (merge request)
Prevent leaking emails of newly created users (merge request)
Added redirect to filtered params (merge request)
Relocate PlantUML config and disable SVG support (merge request)
Sanitize multiple hardlinks from import archives (merge request)
Validates project path availability (merge request)
Fix policy project assign (merge request)
Fix bug where comments on files with incorrect sha breaks UI (merge request)
Fix pipeline schedule authorization for protected branch/tag (merge request)
Mitigate autolink filter ReDOS (merge request)
Fix XSS vector in Web IDE (merge request)
Mitigate project reference filter ReDOS (merge request)
Add a stricter regex for the Harbor search param (merge request)
Update pipeline user to the last policy MR author (merge request)
Prohibit 40 character hex plus a hyphen if branch name is path (merge request)

Non-Security Based Updates

Jenkins 2.417
* Small optimization in computer list.
* Remove the treeview option for artifactList.
* Remove a workaround that was only necessary for OpenJDK 11.0.16 and earlier.
* Use new jenkins-button styling for 'expandableTextbox' button.
* Log agent usage by job.
* Make tab panes accessible via keyboard.
* RPM users with a custom log directory no longer have a logrotate(8) configuration out-of-the-box. (RPM Remove System V initialization script)
* Add allow-same-origin to the sandbox ContentSecurityPolicy directive of workspace and artifact browsers if the Resource Root URL feature is not used. Allow requests to resources like stylesheets and images, even if a reverse proxy prohibits cross-site requests.
* Add the X-Content-Type-Options HTTP header to the response from the agent listener. Silence security scanners that incorrectly report an issue when the HTTP header is missing.
* Only disable the plugin manager "install" button if no plugins are selected (regression in 2.414).

MongoDB 7.0 (Upcoming)
General Changes:
*Cache Refresh Time Fields
* Compound Wildcard Indexes
* Large Change Stream Events
* Store Application Data on Config Shards
* User Roles System Variable
* New Sharding Statistics for Chunk Migrations
* New Slow Query Log Message
* New Parameters

Security:
* Queryable Encryption General Availability
* KMIP 1.0 and 1.1 Support
* Backward-Incompatible Feature

MySQL 8.1
Account Management Notes
Audit Log Notes
Binary Logging
C API Notes
Compilation Notes
Component Notes
Deprecation and Removal Notes
IPv6 Support
Logging Notes
Performance Schema Notes
Spatial Data Support
SQL Syntax Notes
Functionality Added or Changed
Bugs Fixed

Ansible AWX 22.6.0
*Refined release documentation
*Restore pre-upgrade pg_notify notifcation behavior
*Add organization column notification template list
*HostMetricSummaryMonthly command + scheduled task
*Upgrade django to 4.2.3
*Migrate from django-redis to Django's built-in Redis caching support
*Tell Makefile and pre-commit.sh that they are bash
*Allow job_template collection module to set verbosity to 5
*Changing how associations work in awx collection
*Make dispatcher timeout use SIGUSR1, not SIGTERM
*Small doc fixes for workflow and task manager
*Wrap Django RedisCache to mute exceptions
*Require pyyaml >= 6.0.1
*Only push the production images for main repo
*Remove License fields when SUBSCRIPTION_USAGE_MODEL is blank
*Fix collection module docs for names, IDs, and named URLs
*Remove host update code which can be non performant
*Updating release process doc for operator hub instructions
*Add missing trigger for failed-to-start nodes
*Re-enable chdir to project sync to support project-local roles/coll…
*Add a link to EE getting started guide
*Explicitly turn off autocomplete for API login form
*Fix docs link for controller versions >= 4.3
*Only show the product version header when the requester is authenticated
*Add support to collection for named urls
*Simplifications for DependencyManager
*Fix dependencies tag in PR labeler
*Adds autoComplete attribute to forms that were missing it
*Drop unused django-taggit dependency

Strimzi 0.36.1
Important: Strimzi 0.36.1 supports only Kubernetes 1.21 and newer! Kubernetes versions 1.19 and 1.20 are not supported anymore since Strimzi 0.36.
Important: Direct upgrade from Strimzi 0.22 or earlier is not supported anymore!
*Support for Apache Kafka 3.5.1.
*Fix Grafana Dashboards in the Helm Chart.
*Fix issues with 2-node ZooKeeper deployment.
*Documentation fixes.

OpenUpdate - August 3, 2023

Stay Informed

This week, read about:

The Ups and Downs of 0-days: A Year in Review of 0-days Exploited In-the-Wild in 2022.
[systemd-devel] systemd 254 released.
AMD Zenbleed Chip Bug Leaks Secrets Fast and Easy.
'Weird Numerological Coincidence' Found During Work On Linux Kernel 6.5.
ChatGPT Study Suggests Its LLMs Are Getting Dumber At Some Tasks.
Gartner Report: How to Create and Enforce a Governance Policy for Open Source Software.
The Need For a Chief Open Source Officer.
Almost 40% of Ubuntu Users Vulnerable to New Privilege Elevation Flaws.

Key Security, Maintenance, and Features Releases

Security Based Updates

Updates to the OpenLogic CentOS Repository
OpenLogic’s Enterprise Linux Team has recently published the following updates:

CVE-2022-32206
CentOS 8
curl-7.61.1-22_ol001

We recommend that you update your CentOS 8 systems to protect against this vulnerability.

As usual, please ensure that you test these updates before deploying to production.

If you don't currently have CentOS repo access, please reach out to your Perforce/OpenLogic salesperson … you may already be entitled to access with your existing support contract!

Cassandra 4.1.3
New Feature:
* Add a virtual table that exposes currently running queries (CASSANDRA-15241)

Merged from 4.0:
* Revert CASSANDRA-16718 (CASSANDRA-18560)
* Upgrade snappy to 1.1.10.1 (CASSANDRA-18608)
* Fix assertion error when describing mv as table (CASSANDRA-18596)
* Track the amount of read data per row (CASSANDRA-18513)
* Fix Down nodes counter in nodetool describecluster (CASSANDRA-18512)
* Remove unnecessary shuffling of GossipDigests in Gossiper#makeRandomGossipDigest (CASSANDRA-18546)

Merged from 3.11:
* Moved jflex from runtime to build dependencies (CASSANDRA-18664)
* Fix CAST function for float to decimal (CASSANDRA-18647)
* Suppress CVE-2022-45688 (CASSANDRA-18643)
* Remove unrepaired SSTables from garbage collection when only_purge_repaired_tombstones is true (CASSANDRA-14204)
* Wait for live endpoints in gossip waiting to settle (CASSANDRA-18543)
* Fix error message handling when trying to use CLUSTERING ORDER with non-clustering column (CASSANDRA-17818
* Add keyspace and table name to exception message during ColumnSubselection deserialization (CASSANDRA-18346)

Merged from 3.0:
* Suppress CVE-2023-34462 (CASSANDRA-18649)
* Add support for AWS Ec2 IMDSv2 (CASSANDRA-16555)
* Suppress CVE-2023-35116 (CASSANDRA-18630)
* Pass taskId from CompactionTask to system.compaction_history (CASSANDRA-12183)
* Backport CASSANDRA-10508: Remove hard-coded SSL cipher suites (CASSANDRA-18575)
* Suppress CVE-2023-2976 (CASSANDRA-18562)
* Remove dh_python use in Debian packaging (CASSANDRA-18558)

Non-Security Based Updates

Elastic Search 8.9.0
Known Issues

Question Answering fails on long input text. If the context supplied to the task is longer than the model’s max_sequence_length and truncate is set to none then inference fails with the message question answering result has invalid dimension.

Breaking Changes
Aggregations:

Switch TDigestState to use HybridDigest by default

Bug Fixes
Allocation:

Attempt to fix delay allocation
Fix NPE in Desired Balance API
Fix autoexpand during node replace

Authorization:

Resolving wildcard application names without prefix query

CRUD:

Fix retry_on_conflict parameter in update API to not retry indefinitely
Handle failure in TransportUpdateAction#handleUpdateFailureWithRetry

Cluster Coordination:

Avoid getStateForMasterService where possible
Become candidate on publication failure
Fix cluster settings update task acknowledgment

Data streams:

Accept timestamp as object at root level

Geo:

Fix bug when creating empty geo_lines
Fix time-series geo_line to include reduce phase in MergedGeoLines
Support for Byte and Short as vector tiles features

ILM+SLM:

Limit the details field length we store for each SLM invocation

Infra/CLI:

Initialise ES logging in CLI

Infra/Core:

Capture max processors in static init
Interpret microseconds cpu stats from cgroups2 properly as nanos

Infra/Logging:

Add slf4j-nop in order to prevent startup warnings

Infra/REST API:

Fix tchar pattern in RestRequest

Infra/Scripting:

Fix Painless method lookup over unknown super interfaces

Infra/Settings:

Enable validation for versionSettings

Ingest Node:

Fixing DateProcessor when the format is epoch_millis
Fixing GeoIpDownloaderStatsAction$NodeResponse serialization by defensively copying inputs
Trim field references in reroute processor

Machine Learning:

Catch exceptions thrown during inference and report as errors
Fix WordPiece tokenization where stripping accents results in an empty string
Improve model downloader robustness
Prevent high memory usage by evaluating batch inference singularly

Mapping:

Avoid stack overflow while parsing mapping
Fix mapping parsing logic to determine synthetic source is active

Ranking:

Fix sub_searches serialization bug

Recovery:

Promptly fail recovery from snapshot

Search:

Prevent instantiation of top_metrics when sub-aggregations are present
Set new providers before building FetchSubPhaseProcessors

Snapshot/Restore:

Fix blob cache races/assertion errors
Fix reused/recovered bytes for files that are only partially recovered from cache
Fix reused/recovered bytes for files that are recovered from cache
Refactor RestoreClusterStateListener to use ClusterStateObserver

TSDB:

Error message for misconfigured TSDB index
Min score for time series

Task Management:

Improve cancellability in TransportTasksAction

Transform:

Improve reporting status of the transform that is about to finish

Enhancements
Aggregations:

Add cluster setting to SearchExecutionContext to configure TDigestExecutionHint
Add support for dynamic pruning to cardinality aggregations on low-cardinality keyword fields
Make TDigestState configurable
Skip SortingDigest when merging a large digest in HybridDigest
Support value retrieval in top_hits

Allocation:

Take into account expectedShardSize when initializing shard in simulation

Analysis:

Create .synonyms system index

Application:

Add template parameters to Search Applications
Chunk profiling stacktrace response
[Profiling] Add status API
[Profiling] Allow to upgrade managed ILM policy
[Profiling] Introduce ILM for K/V indices
[Profiling] Require POST to retrieve stacktraces
[Profiling] Tweak default ILM policy
[Search Applications] Support arrays in stored mustache templates

Authentication:

Header validator with Security

Authorization:

Add Search ALC filter index prefix to the enterprise search user
Ensure checking application privileges work with nested-limited roles

Autoscaling:

Add shard explain info to ReactiveReason about unassigned shards

DLM:

Add auto force merge functionality to DLM
Adding data_lifecycle to the _xpack/usage API
Adding manage_data_stream_lifecycle index privilege and expanding view_index_metadata for access to data stream lifecycle APIs
Allow for the data lifecycle and the retention to be explicitly nullified

Data streams:

Add support for logs@custom component template for `logs-- data streams
Adding ECS dynamic mappings component and applying it to logs data streams by default
Adjust ECS dynamic templates to support subobjects: false
Automatically parse log events in logs data streams, if their message field contains JSON content
Change default of ignore_malformed to true in logs-*-* data streams
Set @timestamp for documents in logs data streams if missing and add support for custom pipeline
Update data streams implicit timestamp ignore_malformed settings

Engine:

Cache modification time of translog writer file
Trigger refresh when shard becomes search active

Geo:

Add brute force approach to GeoHashGridTiler
Asset tracking - geo_line in time-series aggregations

ILM+SLM:

Chunk the GET _ilm/policy response
Move get lifecycle API to Management thread pool and make cancellable
Reduce WaitForNoFollowersStep requests indices shard stats

Indices APIs:

Bootstrap profiling indices at startup

Infra/Node Lifecycle:

SIGTERM node shutdown type

Ingest Node:

Add mappings for enrich fields
Ingest: expose reroute inquiry/reset via Elastic-internal API bridge

Machine Learning:

Improved compliance with memory limitations
Improve detection of calendar cyclic components with long bucket lengths
Improve detection of time shifts, for example for daylight saving

Mapping:

Allow unsigned long field to use decay functions

Ranking:

Add multiple queries for ranking to the search endpoint

Recovery:

Implement StartRecoveryRequest#getDescription

Search:

Add search shards endpoint
Don’t generate stacktrace in EarlyTerminationException and TimeExceededException
Feature/speed up binary vector decoding
Improve brute force vector search speed by using Lucene functions
Include search idle info to shard stats
Integrate CCS with new search_shards API
Introduce a filtered collector manager
Introduce minimum score collector manager
Skip shards when querying constant keyword fields
Support CCS minimize round trips in async search
Support for patter_replace filter in keyword normalizer
Support null_value for rank_feature field type

Security:

Add "_storage" internal user

Snapshot/Restore:

Reduce overhead in blob cache service get

Stats:

Add ingest information to the cluster info endpoint
Add script information to the cluster info endpoint
Add thread_pool information to the cluster info endpoint

TSDB:

Feature: include unit support for time series rate aggregation

Vector Search

Leverage SIMD hardware instructions in Vector Search

New Features
Application:

Enable analytics geoip in behavioral analytics

Authorization:

Support restricting access of API keys to only certain workflows

Data streams:

Adding ability to auto-install inest pipelines and refer to them from index templates

Geo:

Geometry simplifier

ILM+SLM:

Enhance ILM Health Indicator

Infra/Node Lifecycle:

Gracefully shutdown elasticsearch

Infra/Plugins:

[Fleet] Add .fleet-secrets system index

Machine Learning:

Add support for xlm_roberta tokenized models
Removes the technical preview admonition from query_vector_builder docs

Snapshot/Restore:

Add repo throttle metrics to node stats api response

Stats:

New HTTP info endpoint

Upgrades
Infra/Transport API:

Bump TransportVersion to the first non-release version number. Transport protocol is now versioned independently of release version.

Network:

Upgrade Netty to 4.1.94.Final

Search:

Upgrade Lucene to a 9.7.0 snapshot

Grafana 10.0.3
Features and Enhancements:
*Alerting: Sort NumberCaptureValues in EvaluationString.
*Alerting: No longer silence paused alerts during legacy migration.
*Auth: Add support for custom signing keys in auth.azure_ad.
*Chore: Upgrade Go to 1.20.6.
*Auth: Remove ldap init sync. (Enterprise)
*Chore: Upgrade Go to 1.20.6. (Enterprise)

Bug Fixes:
*Alerting: Fix edit / view of webhook contact point when no authorization is set.
*AzureMonitor: Set timespan in Logs Portal URL link.
*Plugins: Only configure plugin proxy transport once.
*Elasticsearch: Fix multiple max depth flatten of multi-level objects.
*Elasticsearch: Fix histogram colors in backend mode.
*Alerting: Fix state in expressions footer.
*AppChromeService: Fixes update to breadcrumb parent URL.
*Elasticsearch: Fix using multiple indexes with comma separated string.
*Alerting: Fix Alertmanager change detection for receivers with secure settings.
*Transformations: Fix extractFields throwing Error if one value is undefined or null.
*XYChart: Point size editor should reflect correct default (5).
*Annotations: Fix database lock while updating annotations.
*TimePicker: Fix issue with previous fiscal quarter not parsing correctly.
*AzureMonitor: Correctly build multi-resource queries for Application Insights components.
*AzureMonitor: Fix metric names for multi-resources.
*Logs: Do not insert log-line into log-fields in json download.
*Loki: Fix wrong query expression with inline comments.
*License: Enable FeatureUserLimit for all products. (Enterprise)

Jenkins 2.416
*Community reported issues: 1×JENKINS-71699
*Replace browser confirm with modal dialogs in many places.
*Add last build status to job page.
*Remove the rebuild plugin from the setup wizard plugin selection.
*Estimate project duration accurately in more cases (regression in 2.407).
*Developer: API for alert, confirm, prompt, modal and form dialogs
*Remove long deprecated hudson.util.IOUtils#DIR_SEPARATOR, hudson.util.IOUtils#DIR_SEPARATOR_WINDOWS, hudson.util.IOUtils#DIR_SEPARATOR_UNIX, hudson.util.IOUtils#LINE_SEPARATOR,   hudson.util.IOUtils#LINE_SEPARATOR_WINDOWS, and hudson.util.IOUtils#LINE_SEPARATOR_UNIX which are available from org.apache.commons.io.IOUtils.

Kibana 8.9.0
Breaking Changes

Breaking changes can prevent your application from optimal operation and performance. Before you upgrade to 8.9.0, review the breaking changes, then mitigate the impact to your application.
Hide Uptime app if no data is available
Remove synthetics pattern from Uptime settings

Deprecations

The following functionality is deprecated in 8.9.0, and will be removed in 9.0.0. Deprecated functionality does not have an immediate impact on your application, but we strongly recommend you make the necessary updates after you upgrade to 8.9.0.
Hide ability to create legacy input controls
Remove legacy field stats

Features

Kibana 8.9.0 adds the following new and notable features.

APM:

Removes default service name and environment
Adds Agent status action
Added sessionSampleRate to agent configuration, which is a mobile specific setting
Adds storage explorer improvements

Fleet:

Adds CloudFormation install method to CSPM
Adds flags to give permissions to write to any dataset and namespace
Disables Agent ID verification for Observability projects
Setup ignore_malformed in fleet

Lens & Visualizations:

Adds several new capabilities for annotation groups in Lens

Observability:

Adds SLO create callout to service overview, transactions page and transactions details
Adds the Logs threshold alert detail page, which provides more information and context about the Logs threshold alert

Security:

Adds vulnerability dashboard tables
Adds new Vulnerabilities tab to the Group by Resource page
Adds display errors and check licenses for actions in response actions
Adds common response actions tab in the alert flyou

Logstash 8.9.0
Notable Issues Fixed:

Fixed an issue where installs and updates of certain Logstash plugins could fail when located behind a proxy. This issue surfaced after logstash-filter-translate was updated to require that the jar-dependencies gem be used to retrieve artifacts from maven when the plugin was installed. This requirement could prevent the plugin update when a proxy was in use.
Improved logging when Logstash is stalled on shutdown. We now provide additional information about the main thread if it is causing the shutdown to stall.
Improved SSL settings for connection to Elasticsearch for central management and monitoring. This commit adds settings support for file-based certificates and cipher suites for management and monitoring settings, and removes the deprecation warnings from the logs that have been in since SSL configuration settings were revamped in the Elasticsearch output.

Updates to dependencies:

Update Bundler to version 2.4

Plugins:
Azure_event_hubs Input - 1.4.5

Update multiple dependencies such as gson, log4j2, jackson

Beats Input - 6.6.3

[DOC] Updated the ssl_client_authentication and ssl_verify_mode documentation explaining that CN and SAN are not validated.
Update netty to 4.1.94 and jackson to 2.15.2

Http Input - 3.7.2

Update netty to 4.1.94

Snmp Input - 1.3.2

[DOC] Add troubleshooting help for "failed to locate MIB module" error when using smidump to convert MIBs

Tcp Input - 6.3.5

Update netty to 4.1.94 and other dependencies
Fix: reduce error logging (to info level) on connection resets

Tcp Output - 6.1.2

Changed the client mode to write using the non-blocking method.

Prometheus 2.46.0
[FEATURE] Promtool: Add PromQL format and label matcher set/delete commands to promtool.
[FEATURE] Promtool: Add push metrics command.
[ENHANCEMENT] Promtool: Read from stdin if no filenames are provided in check rules.
[ENHANCEMENT] Hetzner SD: Support larger ID's that will be used by Hetzner in September.
[ENHANCEMENT] Kubernetes SD: Add more labels for endpointslice and endpoints role.
[ENHANCEMENT] Kubernetes SD: Do not add pods to target group if the PodIP status is not set.
[ENHANCEMENT] OpenStack SD: Include instance image ID in labels.
[ENHANCEMENT] Remote Write receiver: Validate the metric names and labels.
[ENHANCEMENT] Web: Initialize prometheus_http_requests_total metrics with code label set to 200.
[ENHANCEMENT] TSDB: Add Zstandard compression option for wlog.
[ENHANCEMENT] TSDB: Support native histograms in snapshot on shutdown.
[ENHANCEMENT] Labels: Avoid compiling regexes that are literal.
[BUGFIX] Histograms: Fix parsing of float histograms without zero bucket.
[BUGFIX] Histograms: Fix scraping native and classic histograms missing some histograms.
[BUGFIX] Histograms: Enable ingestion of multiple exemplars per sample.
[BUGFIX] File SD: Fix path handling in File-SD watcher to allow directory monitoring on Windows.
[BUGFIX] Linode SD: Cast InstanceSpec values to int64 to avoid overflows on 386 architecture.
[BUGFIX] PromQL Engine: Include query parsing in active-query tracking.
[BUGFIX] TSDB: Handle TOC parsing failures.

Gitlab 16.2.1
Fixed (1 change)
*Fix crash when LDAP CA file set outside tls_options

Stay Informed about Open Source News and Security Updates

As open source support experts, we monitor community projects to ensure our customers’ environments include the latest releases and are protected against emerging threats. We share what we learn about important open source news including software releases, trending topics, and other related information including upcoming OpenLogic events in our OpenUpdate Weekly newsletter.

Complete the form to receive an email message when we post a new OpenUpdate.

Have Questions or Need Support?

If you have any questions about the content in this week’s newsletter, or are interested in getting support for your open source software, please contact one of our experts.

Talk To An Expert

Learn more about open source including technologies, industry trends, and available services.

See All Resources