Stay Informed

This week, read about:

Key Security, Maintenance, and Features Releases

Security Based Updates

Updates to the OpenLogic CentOS Repository
*) OpenLogic’s Enterprise Linux Team has recently published the following updates:

We recommend that you update your CentOS 6 systems to ensure proper timekeeping. As usual, please ensure that you test these updates before deploying to production. If you don't currently have CentOS repo access, please reach out to your Perforce/OpenLogic salesperson … you may already be entitled to access with your existing support contract!

Solr 9.4.1
Bug Fixes:

  • SOLR-17039: Entropy calculation in bin/solr script fails in Docker due to missing 'bc' cmd
  • SOLR-17057: JSON Query regression: If "query" is specified with a String (not JSON structure), "defType" should parse it. Since 9.4 defType was ignored.
  • SOLR-6853: Allow '/' characters in the text managed by Managed Resources API.
  • SOLR-17060: CoreContainer#create may deadlock with concurrent requests for metrics
  • SOLR-17098: ZK Credentials and ACLs are no longer sent to all ZK Servers when using Streaming Expressions. They will only be used when sent to the default ZK Host.
  • SOLR-16203: Properly initialize schema plugins loaded by SPI name
  • CVE-2023-50290: Apache Solr allows read access to host environment variables

Non-Security Based Updates

MySQL 8.3.6
Audit Log Notes:

Compilation Notes:

  • Microsoft Windows: MySQL did not compile correctly using Visual Studio 2022. (Bug #35967676)
  • Improved the -DWITH_ZLIB=system check. (Bug #35968195)
  • For compiling on Linux, changed the no-error=deprecated-declarations flag to no-deprecated-declarations for the OpenSSL 3 library.

Our thanks to karry zhang for the contribution. (Bug #112209, Bug #35755328)

Optimizer Notes:

  • The hashing algorithm employed yielded poor performance when using a HASH field to check for uniqueness. (Bug #109548, Bug #34959356)

Packaging Notes:

  • Important Change: The GnuPG build key (A8D3785C) used to sign MySQL downloadable packages has been updated. The previous GnuPG build key (3A79BD29) expired on 2023-12-14. For information about verifying the integrity and authenticity of MySQL downloadable packages using GnuPG signature checking, or to obtain a copy of our public GnuPG build key, see Signature Checking Using GnuPG.

Due to the GnuPG key update, systems configured to use repo.mysql.com may report a signature verification error when upgrading to MySQL 8.0.36 and higher or to MySQL 8.3.0 and higher using apt or yum. Use one of the following methods to resolve this issue:

  1. Manually reinstall the MySQL APT or YUM repository setup package from https://dev.mysql.com/downloads/.
  2. Download the MySQL GnuPG public key and add it your system GPG keyring.

Performance Schema Notes:

  • When executing a stored program, the Performance Schema instrumentation caused some unnecessary overhead.

As of this release, all stored procedure micro instructions (statement/sp/%), except statement/sp/stmt, are disabled by default. (Bug #27934653)

  • The performance of the Performance Schema statement instrumentation has been improved. Specifically, collecting MESSAGE_TEXT data is now more efficient. (Bug #112621, Bug #35916912)

Pluggable Authentication:

  • Beginning with this release, the behavior of the AUTHENTICATION_PAM_LOG environment variable used in debugging the PAM authentication plugin is changed as follows:
    • Setting AUTHENTICATION_PAM_LOG to an arbitrary value (except as noted in the next item) no longer includes passwords in its diagnostic messages.
    • To include passwords in the diagnostic messages, set AUTHENTICATION_PAM_LOG=PAM_LOG_WITH_SECRET_INFO.

For more information, see PAM Authentication Debugging. (Bug #74313, Bug #20042010)

Functionality Added or Changed:

  • Important Change: For platforms on which OpenSSL libraries are bundled, the linked OpenSSL library for MySQL Server has been updated to version 3.0.12. Issues fixed in OpenSSL version 3.0.12 are described at https://www.openssl.org/news/cl30.txt. (Bug #36033684)

Bugs Fixed:

  • InnoDB: The hash function used by the adaptive hash index (AHI) was improved to increase performance. (Bug #35449386)
  • InnoDB: If change buffer entries are present during startup, a disabled innodb_validate_tablespace_paths option will no longer be enforced and instead the MySQL server will proceed to validate all tablespaces. Otherwise, secondary indexes could end up corrupted. (Bug #35208990)
  • InnoDB: During concurrent DDL and DML operations, DDL could fail if the online log grew too large. Buffer handling was improved to prevent this issue. (Bug #35115601)
  • Replication: An issue with calculating the current number of bytes used for Log_event events in Performance Schema memory instrumentation made it appear as though the sql/replica_sql thread on the replica grew endlessly and never decreased in size. (Bug #35546877)
  • Replication: Stopping replication while replicating CREATE TABLE AS SELECT caused the server to exit. (Bug #33934013)
  • Group Replication: A forced START GROUP_REPLICATION while a replication channel was in an error state could lead to an unplanned server exit. (Bug #34724344)
  • For building Enterprise Linux RPMs, the build scripts now point to a newer strip command (under /opt/rh/gcc-toolset-12), and they now check that the corresponding dwz tool is available. (Bug #36086236)
  • In some cases, calling a loadable function installed by an improperly initialized plugin caused an unplanned shutdown. (Bug #35889261)
  • Found and fixed an assertion failure at handler::ha_index_end() in handler.cc. (Bug #35877600)
  • When the MYSQL_FIREWALL plugin was configured to use a custom schema, but failed to initialize properly during the server startup, subsequent errors and failures could occur. (Bug #35853298)
  • Some nested queries with GROUP BY were not handled correctly. (Bug #35846402, Bug #35945822)

References: This issue is a regression of: Bug #32918400.

  • In limited cases, passing data to the MD5() encryption function could halt the server. (Bug #35764496)
  • Some subselects from views were not always handled correctly. (Bug #35738548)
  • While performing an operation such as the bulk renaming of many tables, simultaneously executing a data definition statement similar to CREATE TABLE ... SELECT could stop the server unexpectedly. (Bug #35735937)
  • UPDATE HISTOGRAM did not behave as expected in all cases.

UPDATE HISTOGRAM did not behave as expected in all cases. (Bug #35710404)

  • EXPLAIN ANALYZE did not always produce the expected result. (Bug #35710383)
  • An error occurred during subquery resolution. (Bug #35710373)

References: This issue is a regression of: Bug #35184353.

  • Refreshing of used table information is now postponed to the start of the next execution, just after tables have been opened, and we know that all table objects are in a proper state. (Bug #35710213)
  • Some HAVING queries did not produce expected results. (Bug #35710183)
  • Some recursive CTEs did not function as expected. (Bug #35654240)
  • Some queries using OVER (PARTITION ...) were not always executed successfully. (Bug #35627798)
  • Some subqueries with ROLLUP were not always handled correctly. (Bug #35621842, Bug #35804794)
  • (Bug #35529968)
  • Removed the CPACK_COMPONENT_GROUP_INFO_DISPLAY_NAME configuration option from the Windows installation MSI interface. Now the INFO_BIN and INFO_SRC files are always installed. (Bug #35529968)
  • Some queries using windowing functions were not always handled correctly. (Bug #35471471)
  • In debug builds, a case-altered column name could cause the server to exit. (Bug #35449266)
  • The MySQL Server and MySQL Cluster packages contained two copies of the INFO_SRC file. (Bug #35400142)
  • A SELECT statement within a prepared statement unexpectedly returned different results on successive executions. (Bug #35340987, Bug #35846585, Bug #35846873)

References: This issue is a regression of: Bug #35060385.

  • Some SELECT DISTINCT queries were not always handled correctly. (Bug #33725447)
  • Removed an assertion failure in sql/field.cc. (Bug #112503, Bug #35846221)
  • Sme queries having the form SELECT AVG(...) OVER (PARTITION BY ...) were not always handled correctly. (Bug #112460, Bug #35710179, Bug #35845413)
  • Upgrading MySQL using an official MySQL Yum or SUSE repository always enables the MySQL service. Now it enables the service only after installing, and preserves (and does not edit) the existing value while upgrading. (Bug #112382, Bug #35823558)
  • For a query with a derived condition pushdown where a column in the condition needs to be replaced, a matching item could not found, even when known to be present, when the replacement item was wrapped in a ROLLUPwhile the matching item was not. (Bug #111665, Bug #35498378, Bug #35570065, Bug #35826171)

References: This issue is a regression of: Bug #33349994.

  • Performing an arithmetic operation on the result over a window function in a stored procedure gave the correct result the first time the procedure was executed, but returned an incorrect result on all subsequent invocations. (Bug #110983, Bug #35380604)

References: See also: Bug #110847, Bug #35340987.

  • MySQL did not build correctly using the musl version of libc.

Our thanks to Sam James for the contribution. (Bug #110808, Bug #35330950)

  • In some cases, selecting from a view leaked a small amount of memory. (Bug #103133, Bug #32764586) 

Docker Compose 2.24.1
Fixes:

  • Stop the resource timer after last expected event by @ndeloof in #11357
  • fix engine version require to use healthcheck.start_interval by @ndeloof in #11360
  • fix(tracing): batch span exports to prevent blocking by @milas in #11364

Internal:

  • remove watch subcommand from the alpha command by @glours in #11363
  • signals/utils: always handle received signals by @laurazard in #11361

Dependencies:

  • build(deps): bump github.com/containerd/containerd from 1.7.11 to 1.7.12 by @dependabot in #11347
  • build(deps): bump github.com/docker/cli from 25.0.0-rc.1+incompatible to 25.0.0-rc.2+incompatible by @dependabot in #11348
  • build(deps): bump github.com/docker/docker from 25.0.0-rc.1+incompatible to 25.0.0-rc.2+incompatible by @dependabot in #11349
  • build(deps): bump github.com/docker/cli from 25.0.0-rc.2+incompatible to 25.0.0-rc.3+incompatible by @dependabot in #11365
  • build(deps): bump github.com/docker/docker from 25.0.0-rc.2+incompatible to 25.0.0-rc.3+incompatible by @dependabot in #11367
  • bump version of compose-go to v2.0.0-rc.1 by @glours in #11368

Jenkins 2.441

  •  Update operating system end of life data for Amazon Linux, Alpine Linux, and Fedora Linux. (pull 8864)
  •  Remove unused material icons. (pull 8831)
  •  Fix build button rendering for Dashboard View plugin. (pull 8854)
  •  Change focus in the new item page only if from has a valid job name. (issue 66530)

Elasticsearch 8.12.0
Breaking changes:

  • There are no breaking changes in 8.12

Notable changes:
There are notable changes in 8.12 that you need to be aware of but that we do not consider breaking, items that we may consider as notable changes are

  • Changes to features that are in Technical Preview.
  • Changes to log formats.
  • Changes to non-public APIs.
  • Behaviour changes that repair critical bugs.

Authorization:

  • Fixed JWT principal from claims #101333

ES|QL:

  • [ES|QL] pow function always returns double #102183 (issue: #99055)

Infra/Plugins:

  • Remove Plugin.createComponents method in favour of overload with a PluginServices object #101457

Bug fixes
Aggregations:

  • Adjust Histogram’s bucket accounting to be iteratively #102172
  • Aggs error codes part 1 #99963
  • Skip global ordinals loading if query does not match after rewrite #102844
  • Trigger parent circuit breaker when building scorers in filters aggregation #102511
  • Unwrap ExecutionException when loading from cache in AbstractIndexOrdinalsFieldData #102476

Application:

  • [Connector API] Fix bug in configuration validation parser #104198
  • [Connector API] Fix bug with nullable tooltip field in parser #103427
  • [Connectors API] Fix ClassCastException when creating a new sync job #103508
  • [Connectors API] Fix bug with missing TEXT DisplayType enum #103430
  • [Connectors API] Handle nullable fields correctly in the ConnectorSyncJob parser #103183
  • [Profiling] Query in parallel only if beneficial #103061
  • [Search Applications] Return 400 response when template rendering produces invalid JSON #101474

Authentication:

  • Fall through malformed JWTs to subsequent realms in the chain #101660 (issue: #101367)

Authorization:

  • Fix cache invalidation on privilege modification #102193

Data streams:

  • Use dataset size instead of on-disk size for data stream stats #103342

Distributed:

  • Active shards message corrected for search shards #102808 (issue: #101896)
  • Dispatch ClusterStateAction#buildResponse to executor #103435
  • Fix listeners in SharedBlobCacheService.readMultiRegions #101727

Downsampling:

  • Copy counter field properties to downsampled index #103580 (issue: #103569)
  • Fix downsample api by returning a failure in case one or more downsample persistent tasks failed #103615

EQL:

  • Cover head/tail commands edge cases and data types coverage #101859 (issue: #101724)
  • Samples should check if the aggregations result is empty or null #103574

ES|QL:

  • ESQL: Fix to_degrees() returning infinity #103209 (issue: #102987)
  • ESQL: Fix planning of MV_EXPAND with foldable expressions #101385 (issue: #101118)
  • ESQL: Fix rare bug with empty string #102350 (issue: #101969)
  • ESQL: Fix resolution of MV_EXPAND after KEEP * #103339 (issue: #103331)
  • ESQL: Fix single value query #102317 (issue: #102298)
  • ESQL: Improve local folding of aggregates #103670
  • ESQL: Improve pushdown of certain filters #103671
  • ESQL: Narrow catch in convert functions #101788 (issue: #100820)
  • ESQL: Update the use of some user-caused exceptions #104046
  • ESQL: remove time_zone request parameter #102767 (issue: #102159)
  • ES|QL: Fix NPE on single value detection #103150 (issue: #103141)
  • ES|QL: Improve resolution error management in mv_expand #102967 (issue: #102964)
  • Fix layout for MV_EXPAND #102916 (issue: #102912)
  • Fix planning of duplicate aggs #102165 (issue: #102083)
  • AsyncOperator#isFinished must never return true on failure #104029

Engine:

  • Fix lastUnsafeSegmentGenerationForGets for realtime get #101700

Geo:

  • Fix geo tile bounding boxes to be consistent with arithmetic method #100826 (issues: #92611, #95574)

ILM+SLM:

  • Collect data tiers usage stats more efficiently #102140 (issue: #100230)

Indices APIs:

  • Fix template simulate setting application ordering #103024 (issue: #103008)

Infra/Core:

  • Cache component versions #103408 (issue: #102103)
  • Fix metric gauge creation model #100609

Infra/Node Lifecycle:

  • Wait for reroute before acking put-shutdown #103251

Infra/Plugins:

  • Making classname optional in Transport protocol #99702 (issue: #98584)

Infra/Scripting:

  • Make IPAddress writeable #101093 (issue: #101082)

Infra/Settings:

  • Report full stack trace for non-state file settings transforms #101346

Ingest Node:

  • Sending an index name to DocumentParsingObserver that is not ever null #100862

License:

  • Error log when license verification fails locally #102919

Machine Learning:

  • Catch exceptions during pytorch_inference startup #103873
  • Ensure the estimated latitude is within the allowed range #2586
  • Exclude quantiles when fetching model snapshots where possible #103530
  • Fix frequent_item_sets aggregation on empty index #103116 (issue: #103067)
  • If trained model download task is in progress, wait for it to finish before executing start trained model deployment #102944
  • Persist data counts on job close before results index refresh #101147
  • Preserve response headers in Datafeed preview #103923
  • Prevent attempts to access non-existent node information during rebalancing #103361
  • Prevent resource over-subscription in model allocation planner #100392
  • Remove dependency on the IPEX library #2605 and #2606
  • Start a new trace context before loading a trained model #103124
  • Wait for the model results on graceful shutdown #103591 (issue: #103414)

Monitoring:

  • [Monitoring] Dont get cluster state until recovery #100565

Network:

  • Ensure the correct threadContext for RemoteClusterNodesAction #101050

Ranking:

  • Add an additional tiebreaker to RRF #101847 (issue: #101232)

Reindex:

  • Allow prefix index naming while reindexing from remote #96968 (issue: #89120)

Search:

  • Add JIT compiler excludes for computeCommonPrefixLengthAndBuildHistogram #103112
  • Check that scripts produce correct json in render template action #101518 (issue: #101477)
  • Fix NPE & empty result handling in CountOnlyQueryPhaseResultConsumer #103203
  • Fix format string in OldLuceneVersions #103185
  • Handle timeout on standalone rewrite calls #103546
  • Introduce Elasticsearch PostingFormat based on Lucene 90 positing format using PFOR #103601 (issue: #103002)
  • Restore inter-segment search concurrency with synthetic source is enabled #103690
  • Support complex datemath expressions in index and index alias names #100646

Snapshot/Restore:

  • More consistent logging messages for snapshot deletion #101024
  • Reroute on shard snapshot completion #101585 (issue: #101514)

TSDB:

  • Throw when wrapping rate agg in DeferableBucketAggregator #101032

Transform:

  • Add an assertion to the testTransformFeatureReset test case #100287
  • Consider search context missing exceptions as recoverable #102602
  • Consider task cancelled exceptions as recoverable #100828
  • Fix NPE that is thrown by _update API #104051 (issue: #104048)
  • Log stacktrace together with log message in order to help debugging #101607
  • Split comma-separated source index strings into separate indices #102811 (issue: #99564)

Vector Search:

  • Disallow vectors whose magnitudes will not fit in a float #100519

Watcher:

  • Correctly logging watcher history write failures #101802

Enhancements
Aggregations:

  • Check the real memory circuit breaker when building global ordinals #102462
  • Disable concurrency for sampler and diversified sampler #102832
  • Disable parallelism for composite agg against high cardinality fields #102644
  • Enable concurrency for multi terms agg #102710
  • Enable concurrency for scripted metric agg #102461
  • Enable inter-segment concurrency for terms aggs #101390
  • Export circuit breaker trip count as a counter metric #101423
  • Introduce fielddata cache ttl #102682
  • Status codes for Aggregation errors, part 2 #100368
  • Support keyed histograms #101826 (issue: #100242)

Allocation:

  • Add more desired balance stats #102065
  • Add undesired shard count #101426
  • Expose reconciliation metrics via APM #102244

Application:

  • Calculate CO2 and emmission and costs #101979
  • Consider duplicate stacktraces in custom index #102292
  • Enable Universal Profiling as Enterprise feature #100333
  • Include totals in flamegraph response #101126
  • Retrieve stacktrace events from a custom index #102020
  • [Profiling] Notify early about task cancellation #102740
  • [Profiling] Report in status API if docs exist #102735

Authentication:

  • Add ldap user metadata mappings for full name and email #102925
  • Add manage_enrich cluster privilege to kibana_system role #101682

Authorization:

  • Remove auto_configure privilege for profiling #101026
  • Use BulkRequest to store Application Privileges #102056
  • Use non-deprecated SAML callback URL in SAML smoketests #99983 (issue: #99986)
  • Use non-deprecated SAML callback URL in tests #99983 (issue: #99985)

CAT APIs:

  • Expose roles by default in cat allocation API #101753

CRUD:

  • Cache resolved index for mgets #101311

Data streams:

  • Introduce new endpoint to expose data stream lifecycle stats #101845
  • Switch logs data streams to search all fields by default #102456 (issue: #99872)

Distributed:

  • Add support for configuring proxy scheme in S3 client settings and EC2 discovery plugin #102495 (issue: #101873)
  • Introduce a StreamOutput that counts how many bytes are written to the stream #102906
  • Push s3 requests count via metrics API #100383
  • Record operation purpose for s3 stats collection #100236

EQL:

  • Add error logging for *QL #101057
  • Use the eql query filter for the open-pit request #103212

ES|QL:

  • ESQL: Add profile option #102713
  • ESQL: Alias duplicated aggregations in a stats #100642 (issue: #100544)
  • ESQL: Load more than one field at once #102192
  • ESQL: Load stored fields sequentially #102727
  • ESQL: Load text field from parent keyword field #102490 (issue: #102473)
  • ESQL: Make blocks ref counted #100408
  • ESQL: Make fieldcaps calls lighter #102510 (issues: #101763, #102393)
  • ESQL: More tracking in BlockHash impls #101488
  • ESQL: New telemetry commands #102937
  • ESQL: Share constant null Blocks #102673
  • ESQL: Short circuit loading empty doc values #102434
  • ESQL: Support the _source metadata field #102391
  • ESQL: Track blocks emitted from lucene #101396
  • ESQL: Track memory from values loaded from lucene #101383
  • Fast path for reading single doc with ordinals #102902
  • Introduce local block factory #102901
  • Load different way #101235
  • Track ESQL enrich memory #102184
  • Track blocks in AsyncOperator #102188
  • Track blocks of intermediate state of aggs #102562
  • Track blocks when hashing single multi-valued field #102612
  • Track pages in ESQL enrich request/response #102190

Engine:

  • Add static node settings to set default values for max merged segment sizes #102208

Geo:

  • Add runtime field of type geo_shape #100492 (issue: #61299)

Health:

  • Add message field to HealthPeriodicLogger and S3RequestRetryStats #101989
  • Add non-green indicator names to HealthPeriodicLogger message #102245

ILM+SLM:

  • Health Report API should not return RED for unassigned cold/frozen shards when data is available #100776
  • Switch fleet’s built-in ILM policies to use .actions.rollover.max_primary_shard_size #99984 (issue: #99983)

Indices APIs:

  • Add executed pipelines to bulk api response #100031
  • Add support for marking component templates as deprecated #101148 (issue: #100992)
  • Allowing non-dynamic index settings to be updated by automatically unassigning shards #101723
  • Rename component templates and pipelines according to the new naming conventions #99975
  • Run TransportGetAliasesAction on local node #101815

Infra/CLI:

  • Set ActiveProcessorCount when node.processors is set #101846

Infra/Core:

  • Add apm api for asynchronous counters (always increasing) #102598
  • Log errors in RestResponse regardless of error_trace parameter #101066 (issue: #100884)

Infra/Logging:

  • Add status code to rest.suppressed log output #100990

Ingest Node:

  • Deprecate the unused elasticsearch_version field of enrich policy json #103013
  • Optimize MurmurHash3 #101202

Machine Learning:

  • Accept a single or multiple inputs to _inference #102075
  • Add basic telelemetry for the inference feature #102877
  • Add internal inference action for ml models an services #102731
  • Add prefix strings option to trained models #102089
  • Estimate the memory required to deploy trained models more accurately #98874
  • Improve stability of spike and dip detection for the change point aggregation #102637
  • Include ML processor limits in _ml/info response #101392
  • Read scores from downloaded vocabulary for XLM Roberta tokenizers #101868
  • Support for GET all models and by task type in the _inference API #102806
  • Upgrade Boost libraries to version 1.83 #2560

Mapping:

  • Improve analyzer reload log message #102273

Monitoring:

  • Add memory utilization Kibana metric to the monitoring index templates #102810
  • Added beat.stats.libbeat.pipeline.queue.max_events #102570

Network:

  • Record more detailed HTTP stats #99852

Search:

  • Add metrics to the shared blob cache #101577
  • Add support for Serbian Language Analyzer #100921
  • Add support for index_filter to open pit #102388 (issue: #99740)
  • Added metric for cache eviction of entries with non zero frequency #100570
  • Disable inter-segment concurrency when sorting by field #101535
  • Enable query phase parallelism within a single shard #101230 (issue: #80693)
  • Node stats as metrics #102248
  • Optimize _count type API requests #102888

Security:

  • Expose the invalidation field in Get/Query ApiKey APIs #102472
  • Make api_key.delete.interval a dynamic setting #102680

Snapshot/Restore:

  • Fail S3 repository analysis on partial reads #102840
  • Parallelize stale index deletion #100316 (issue: #61513)
  • Repo analysis of uncontended register behaviour #101185
  • Repo analysis: allow configuration of register ops #102051
  • Repo analysis: verify empty register #102048

Stats:

  • Introduce includeShardsStats in the stats request to indicate that we only fetch a summary #100466 (issue: #99744)
  • Set includeShardsStats = false in NodesStatsRequest where the caller does not use shards-level statistics #100938

Store:

  • Add methods for adding generation listeners with primary term #100899
  • Allow executing multiple periodic flushes while they are being made durable #102571
  • Pass shard’s primary term to Engine#addSegmentGenerationListener #99752

Transform:

  • Implement exponential backoff for transform state persistence retrying #102512 (issue: #102528)
  • Make tasks that calculate checkpoints time out #101055
  • Pass source query to _field_caps (as index_filter) when deducing destination index mappings for better performance #102379
  • Pass transform source query as index_filter to open_point_in_time request #102447 (issue: #101049)
  • Skip shards that don’t match the source query during checkpointing #102138

Vector Search:

  • Add vector_operation_count in profile output for knn searches #102032
  • Make cosine similarity faster by storing magnitude and normalizing vectors #99445

New features
Application:

  • Enable Connectors API as technical preview #102994
  • [Behavioral Analytics] Analytics collections use Data Stream Lifecycle (DSL) instead of Index Lifecycle Management (ILM) for data retention management. Behavioral analytics has traditionally used ILM to manage data retention. Starting with 8.12.0, this will change. Analytics collections created prior to 8.12.0 will continue to use their existing ILM policies, but new analytics collections will be managed using DSL. #100033

Authentication:

  • Patterns support for allowed subjects by the JWT realm #102426

Cluster Coordination:

  • Add a node feature join barrier. This prevents nodes from joining clusters that do not have all the features already present in the cluster. This ensures that once a features is supported by all the nodes in a cluster, that feature will never then not be supported in the future. This is the corresponding functionality for the version join barrier, but for features #101609

Data streams:

  • Add ability to create a data stream failure store #99134

ES|QL:

  • ESQL: emit warnings from single-value functions processing multi-values #102417 (issue: #98743)
  • GEO_POINT and CARTESIAN_POINT type support #102177

Infra/Core:

  • Create new cluster state API for querying features present on a cluster #100974

Ingest Node:

  • Adding a simulate ingest api #101409

Security:

  • Allow granting API keys with JWT as the access_token #101904

Vector Search:

  • Add byte quantization for float vectors in HNSW #102093
  • Make knn search a query #98916

Regressions
Infra/Core:

  • Revert non-semantic NodeInfo #102636

Kibana 8.12.0
Breaking changes

  • Breaking changes can prevent your application from optimal operation and performance. Before you upgrade to 8.12.0, review the breaking changes, then mitigate the impact to your application.

Features:

  • Kibana 8.12.0 adds the following new and notable features.

Alerting:

  • The case list filter bar is now customizable, filters are removable and custom fields can be used as filters (#172276).

APM:

  • Adds viewInApp URL to the custom threshold rule type (#171985).
  • Adds back the mobile crashes & errors tab (#165892).

Elastic Security:

For the Elastic Security 8.12.0 release information, refer to Elastic Security Solution Release Notes.

Elastic Search:

  • Display E5 multilingual callout (#171887).
  • Replace model selection dropdown with list (#171436).

Fleet:

  • Adds support for preconfigured output secrets (Scrypt edition) (#172041).
  • Adds UI components to create and edit output secrets (#169429).
  • Adds support for remote ES output (#169252).
  • Adds the ability to specify secrets in outputs (#169221).
  • Adds an integrations configs tab to display input templates (#168827).
  • Adds a Kibana task to publish Agent metrics (#168435).

Lens & Visualizations:

  • Adds the ability to edit charts made by ES|QL queries in Dashboard (#169911).

Machine Learning:

  • Adds E5 model configurations (#172053).
  • Adds the ability to create a categorization anomaly detection job from pattern analysis (#170567).
  • Adds and displays alerts data in the Anomaly Explorer (#167998).

Observability:

  • Adds logic to update flyout highlights (#172193).
  • Adds logic to display highlights in the flyout (#170650).
  • Changes the Custom threshold title to Beta (#172360).

Security:

  • Disables the connector parameters field (#173610).
  • Adds a risk engine missing privileges callout (#171250).
  • Asset criticality privileges API (#172441).

Uptime:

  • Global params Public APIs (#169669).
  • Private location public API’s (#169376).
  • Settings public API (#163400).

Logstash 8.12
New features and enhancements:

  • Add support for adding and removing multiple keystore keys in a single operation #15739
  • Docker: Update Iron Bank base image to ubi9.2 #15490
  • Internal: extract GeoIP database manager to stand-alone feature #15348

Notable issues fixed:

  • Add missing method of logger wrapper for puma #15640
  • Fix logstash-keystore multiple keys operations with command flags #15737
  • Separate scheduling of segments flushes from time #15697
  • Add system properties to configure Jackson’s stream read constraints #15763
  • Fix issue with Jackson 2.15: Can not write a field name, expecting a value #15564

Updates to dependencies:

  • Add bigdecimal > 3.1 dependency. #15384
  • Update Guava dependency to 32.1.2 #15394
  • Swap dataformat-yaml with snakeyaml #15606
  • Bump Puma to 6.4.2+ #15776
  • Update jackson to 2.15.3 #15477

Documentation enhancements:

  • Add info and link to Logstash running on a Kubernetes cluster through Elastic Cloud on Kubernetes (ECK) #15565
  • Add info for sending Logstash monitoring data to Elastic serverless #15636
  • Add docs for extending integrations with filter-elastic_integration #15674
  • Update Logstash intro and security overview for serverless #15663
  • Update the Logstash-to-Logstash communication docs to reflect the multiple hosts usage #15512

Plugins
Elasticsearch Input - 4.19.1:

  • Plugin version bump to pick up docs fix in #199 required to clear build error in docgen. #200
  • Add search_api option to support search_after and scroll #198
  • The default value auto uses search_after for Elasticsearch >= 8, otherwise, fall back to scroll

Http Input - 3.8.0:

  • Fixed SSL Java KeyStore support #171
  • Added ssl_keystore_type configuration
  • Added SSL Java TrustStore configurations (ssl_truststore_type, ssl_truststore_path and ssl_truststore_password)

Elastic_enterprise_search Integration - 3.0.0:

  • [BREAKING] Swiftype endpoints are no longer supported for both plugins App Search and Workplace Search
  • Bumped Enterprise Search clients to version >= 7.16, < 9 #18
  • Added support to SSL configurations (ssl_certificate_authorities, ssl_truststore_path, ssl_truststore_password, ssl_truststore_type, ssl_verification_mode, ssl_supported_protocols and ssl_cipher_suites)
  • The App Search deprecated options host and path were removed

Kafka Integration - 11.3.3:

  • Fixed: "Can’t modify frozen string" error when record value is nil (tombstones) #155

Logstash Integration - 1.0.1:

  • Fixed: improves throughput by allowing pipeline workers to share a plugin instance concurrently instead of sequentially #19
  • Introduced load balancing mechanism to distribute the requests among the hosts #16

Elasticsearch Output - 11.22.2:

  • Fixed: avoid to populate version and version_type attributes when processing integration metadata and datastream is enabled. #1161
  • Added support for propagating event processing metadata when this output is downstream of an Elastic Integration Filter and configured without explicit version, version_type, or routing directives #1158
  • Added support for propagating event processing metadata when this output is downstream of an Elastic Integration Filter and configured without explicit index, document_id, or pipeline directives #1155
  • Changed the register to initiate pipeline shutdown upon bootstrap failure instead of simply logging the error #1151
  • Doc: Replace document_already_exist_exception with version_conflict_engine_exception in the silence_errors_in_log setting example #1159
  • Doc: Add content for sending data to Elasticsearch on serverless #1164

Kubernetes 1.29.1
API Change:

  • Fixes accidental enablement of the new alpha optionalOldSelf API field in CustomResourceDefinition validation rules, which should only be allowed to be set when the CRDValidationRatcheting feature gate is enabled. Existing CustomResourceDefinition objects which have the field set will retain it on update, but new CustomResourceDefinition objects will not be permitted to set the field while the CRDValidationRatcheting feature gate is disabled. (#122343, @jpbetz) [SIG API Machinery]

Feature:

  • Kubernetes is now built with Go 1.21.6 (#122711, @cpanato) [SIG Release and Testing]

Bug or Regression:

  • Allow deletion of pods that use raw block volumes on node reboot (#122211, @gnufied) [SIG Node and Storage]
  • Fix an issue where kubectl apply could panic when imported as a library (#122559, @Jefftree) [SIG CLI]
  • Fix: Mount point may become local without calling NodePublishVolume after node rebooting. (#119923, @cvvz) [SIG Node and Storage]
  • Fixed a regression since 1.24 in the scheduling framework when overriding MultiPoint plugins (e.g. default plugins). The incorrect loop logic might lead to a plugin being loaded multiple times, consequently preventing any Pod from being scheduled, which is unexpected. (#122366, @caohe) [SIG Scheduling]
  • Fixed migration of in-tree vSphere volumes to the CSI driver. (#122341, @jsafrane) [SIG Storage]
  • QueueingHint implementation for NodeAffinity is reverted because we found potential scenarios where events that make Pods schedulable could be missed. (#122327, @sanposhiho) [SIG Scheduling]
  • QueueingHint implementation for NodeUnschedulable is reverted because we found potential scenarios where events that make Pods schedulable could be missed. (#122326, @sanposhiho) [SIG Scheduling]

Other (Cleanup or Flake):

  • Reverts the EventedPLEG feature (beta, but disabled by default) back to alpha for a known issue (#122718, @pacoxu) [SIG Node]

Node.js 21.6.0
New connection attempt events
Three new events were added in the net.createConnection flow:

  • connectionAttempt: Emitted when a new connection attempt is established. In case of Happy Eyeballs, this might emitted multiple times.
  • connectionAttemptFailed: Emitted when a connection attempt failed. In case of Happy Eyeballs, this might emitted multiple times.
  • connectionAttemptTimeout: Emitted when a connection attempt timed out. In case of Happy Eyeballs, this will not be emitted for the last attempt. This is not emitted at all if Happy Eyeballs is not used.
  • Additionally, a previous bug has been fixed where a new connection attempt could have been started after a previous one failed and after the connection was destroyed by the user. This led to a failed assertion.
  • Contributed by Paolo Insogna in #51045.

Changes to the Permission Model:

  • Node.js 21.6.0 comes with several fixes for the experimental permission model and two new semver-minor commits. We're adding a new flag --allow-addons to enable addon usage when using the Permission Model. 
  • $ node --experimental-permission --allow-addons
  • Contributed by Rafael Gonzaga in #51183
  • And relative paths are now supported through the --allow-fs-* flags. Therefore, with this release one can use:
  • $ node --experimental-permission --allow-fs-read=./index.js
  • To give only read access to the entrypoint of the application.
  • Contributed by Rafael Gonzaga and Carlos Espa in #50758
  • Support configurable snapshot through --build-snapshot-config flag
  • We are adding a new flag --build-snapshot-config to configure snapshots through a custom JSON configuration file.
  • $ node --build-snapshot-config=/path/to/myconfig.json
  • When using this flag, additional script files provided on the command line will not be executed and instead be interpreted as regular command line arguments.
  • These changes were contributed by Joyee Cheung and Anna Henningsen in #50453

Other Notable Changes:

  • [c31ed51373] - (SEMVER-MINOR) timers: export timers.promises (Marco Ippolito) #51246

PHP 8.3.2
Core:

  • Fixed bug GH-12953 (false positive SSA integrity verification failed when loading composer classmaps with more than 11k elements).
  • Fixed bug GH-12999 (zend_strnlen build when strnlen is unsupported).
  • Fixed bug GH-12966 (missing cross-compiling 3rd argument so Autoconf doesn't emit warnings).
  • Fixed bug GH-12854 (8.3 - as final trait-used method does not correctly report visibility in Reflection).

Cli:

  • Fix incorrect timeout in built-in web server when using router script and max_input_time.

DOM:

  • Fixed bug GH-12870 (Creating an xmlns attribute results in a DOMException).
  • Fix crash when toggleAttribute() is used without a document.
  • Fix crash in adoptNode with attribute references.
  • Fixed bug GH-13012 (DOMNode::isEqualNode() is incorrect when attribute order is different).

FFI:

  • Fixed bug GH-9698 (stream_wrapper_register crashes with FFI\CData).
  • Fixed bug GH-12905 (FFI::new interacts badly with observers).

Intl:

  • Fixed GH-12943 (IntlDateFormatter::__construct accepts 'C' as valid locale).

Hash:

  • Fixed bug GH-12936 (hash() function hangs endlessly if using sha512 on strings >= 4GiB).

ODBC:

  • Fix crash on Apache shutdown with persistent connections.

Opcache:

  • Fixed oss-fuzz #64727 (JIT undefined array key warning may overwrite DIM with NULL when DIM is the same var as result).
  • Added workaround for SELinux mprotect execheap issue. See https://bugzilla.kernel.org/show_bug.cgi?id=218258.

OpenSSL:

  • Fixed bug GH-12987 (openssl_csr_sign might leak new cert on error).

PDO:

  • Fix GH-12969 (Fixed PDO::getAttribute() to get PDO::ATTR_STRINGIFY_FETCHES).

PDO_ODBC:

  • Fixed bug GH-12767 (Unable to turn on autocommit mode with setAttribute()).

PGSQL:

  • Fixed auto_reset_persistent handling and allow_persistent type.
  • Fixed bug GH-12974 (Apache crashes on shutdown when using pg_pconnect()).

Phar:

  • Fixed bug #77432 (Segmentation fault on including phar file).

PHPDBG:

  • Fixed bug GH-12962 (Double free of init_file in phpdbg_prompt.c).

SimpleXML:

  • Fix getting the address of an uninitialized property of a SimpleXMLElement resulting in a crash.
  • Fixed bug GH-12929 (SimpleXMLElement with stream_wrapper_register can segfault).

Tidy:

  • Fixed bug GH-12980 (tidynode.props.attribute is missing "Boolean Attributes" and empty attributes).

Prometheus 2.49.0 and 2.49.1
[FEATURE] Promtool: Add --run flag promtool test rules command. #12206
[FEATURE] SD: Add support for NS records to DNS SD. #13219
[FEATURE] UI: Add heatmap visualization setting in the Graph tab, useful histograms. #13096 #13371
[FEATURE] Scraping: Add scrape_config.enable_compression (default true) to disable gzip compression when scraping the target. #13166
[FEATURE] PromQL: Add a promql-experimental-functions feature flag containing some new experimental PromQL functions. #13103 NOTE: More experimental functions might be added behind the same feature flag in the future. Added functions:
Experimental mad_over_time (median absolute deviation around the median) function. #13059
Experimental sort_by_label and sort_by_label_desc functions allowing sorting returned series by labels. #11299
[FEATURE] SD: Add __meta_linode_gpus label to Linode SD. #13097
[FEATURE] API: Add exclude_alerts query parameter to /api/v1/rules to only return recording rules. #12999
[FEATURE] TSDB: --storage.tsdb.retention.time flag value is now exposed as a prometheus_tsdb_retention_limit_seconds metric. #12986
[FEATURE] Scraping: Add ability to specify priority of scrape protocols to accept during scrape (e.g. to scrape Prometheus proto format for certain jobs). This can be changed by setting global.scrape_protocols and scrape_config.scrape_protocols. #12738
[ENHANCEMENT] Scraping: Automated handling of scraping histograms that violate scrape_config.native_histogram_bucket_limit setting. #13129
[ENHANCEMENT] Scraping: Optimized memory allocations when scraping. #12992
[ENHANCEMENT] SD: Added cache for Azure SD to avoid rate-limits. #12622
[ENHANCEMENT] TSDB: Various improvements to OOO exemplar scraping. E.g. allowing ingestion of exemplars with the same timestamp, but with different labels. #13021
[ENHANCEMENT] API: Optimize /api/v1/labels and /api/v1/label/<label_name>/values when 1 set of matchers are used. #12888
[ENHANCEMENT] TSDB: Various optimizations for TSDB block index, head mmap chunks and WAL, reducing latency and memory allocations (improving API calls, compaction queries etc). #12997 #13058 #13056 #13040
[ENHANCEMENT] PromQL: Optimize memory allocations and latency when querying float histograms. #12954
[ENHANCEMENT] Rules: Instrument TraceID in log lines for rule evaluations. #13034
[ENHANCEMENT] PromQL: Optimize memory allocations in query_range calls. #13043
[ENHANCEMENT] Promtool: unittest interval now defaults to evaluation_intervals when not set. #12729
[BUGFIX] SD: Fixed Azure SD public IP reporting #13241
[BUGFIX] API: Fix inaccuracies in posting cardinality statistics. #12653
[BUGFIX] PromQL: Fix inaccuracies of histogram_quantile with classic histograms. #13153
[BUGFIX] TSDB: Fix rare fails or inaccurate queries with OOO samples. #13115
[BUGFIX] TSDB: Fix rare panics on append commit when exemplars are used. #13092
[BUGFIX] TSDB: Fix exemplar WAL storage, so remote write can send/receive samples before exemplars. #13113
[BUGFIX] Mixins: Fix url filter on remote write dashboards. #10721
[BUGFIX] PromQL/TSDB: Various fixes to float histogram operations. #12891 #12977 #12609 #13190 #13189 #13191 #13201 #13212 #13208
[BUGFIX] Promtool: Fix int32 overflow issues for 32-bit architectures. #12978
[BUGFIX] SD: Fix Azure VM Scale Set NIC issue. #13283
[BUGFIX] TSDB: Fixed a wrong q= value in scrape accept header #13313

Spring boot 3.2.2

  • SslBundle implementations do not provide useful toString() results #39167
  • JarEntry.getComment() returns incorrect result from NestedJarFile instances #39166
  • Mixing PEM and JKS certificate material in server.ssl properties does not work #39158
  • Having AspectJ and Micrometer on the classpath is not a strong enough signal to enable support for Micrometer observation annotations #39128
  • Actuator endpoints with no operations that use selectors are not accessible when mapped to / #39122
  • Spring Boot 3.2 app that uses WebFlux, Security, and Actuator may fail to start due to a missing authentication manager #39096
  • management.observations.http.server.requests.name no longer has any effect #39083
  • spring.rabbitmq.listener.stream.auto-startup property has no effect #39078
  • Error mark in the log message for PatternParseException is in the wrong place #39075
  • Configuring server.jetty.max-connections has no effect #39052
  • @ConfigurationPropertiesBinding converters that rely on initial CharSequence to String conversion no longer work #39051
  • Manifest attributes cannot be resolved with the new loader implementation #38996
  • Throwable from logging system initialization may result in the application silently failing to start #38963
  • When using Jetty, idle timeout for IO operations and delayed dispatch cannot be set to less than 30000ms #38960
  • spring-boot-maven-plugin repackage uber jar execution fails when jar is put on WSL network drive #38956
  • Oracle OJDBC BOM version is flagged not for production use #38943
  • Connection leak when using jOOQ and spring.jooq.sql-dialect has not been set #38924
  • AutoConfigurationSorter does not always respect @AutoConfigureOrder(Ordered.LOWEST_PRECEDENCE) #38916
  • Containers are not started when using @ImportTestcontainers #38913
  • Even when spring.security.user.name or spring.security.user.password has been configured, user details auto-configuration still backs off when resource server is on the classpath #38864
  • MockRestServiceServerAutoConfiguration with RestTemplate and RestClient together throws incorrect exception #38820

View all OpenUpdate editions >