In 2015, we launched Amazon Elasticsearch Service (Amazon ES), a fully managed service that makes it easy for you to perform interactive log analytics, real-time application monitoring, website search, and more.

Amazon ES has been a popular service for log analytics because of its ability to ingest high volumes of log data. Additionally, with UltraWarm and cold storage tiers, you can lower costs to one-tenth of traditional hot storage on Amazon ES. Because Amazon ES integrates with Logstash, Amazon Kinesis Data Firehose, Amazon CloudWatch Logs, and AWS IoT, you can select the secure data ingestion tool that meets your use case requirements.

Developers embrace open-source software for many reasons. One of the most important reasons is the freedom to use that software where and how they want. On January 21, 2021, Elastic NV announced that they would change their software licensing strategy. After Elasticsearch version 7.10.2 and Kibana 7.10.2, they will not release new versions of Elasticsearch and Kibana under the permissive 2.0 version of the Apache License (ALv2). Instead, Elastic NV is releasing Elasticsearch and Kibana under the Elastic license, with source code available under the Elastic License or Server Side Public License (SSPL). These licenses are not open source and do not offer users the same freedom.

For this reason, we decided to create and maintain OpenSearch, a community-driven, open-source fork from the last ALv2 version of Elasticsearch and Kibana. We are making a long-term investment in the OpenSearch project and recently released version 1.0.

OpenSearch provides a highly scalable system for providing fast access and response to large volumes of data with an integrated visualization tool, OpenSearch Dashboards, that makes it easy for users to explore their data. OpenSearch and OpenSearch Dashboards were originally derived from Elasticsearch 7.10.2 and Kibana 7.10.2. Like Elasticsearch and Apache Solr, OpenSearch is powered by the Apache Lucene search library.

Announcing Amazon OpenSearch Service
Today, we rename Amazon Elasticsearch Service to Amazon OpenSearch Service because the service now supports OpenSearch 1.0. Although the name has changed, we will continue to deliver the same experiences without any negative impact to ongoing operations, development methodology, or business use.

2021 opensearch service 1

Amazon OpenSearch Service offers a choice of open-source engines to deploy and run, including the currently available 19 versions of ALv2 Elasticsearch 7.10 and earlier and OpenSearch 1.0. We will continue to support and maintain the ALv2 Elasticsearch versions with security and bug fixes. We will deliver all-new features and functionality through OpenSearch and OpenSearch Dashboards. Amazon OpenSearch Service APIs will be backward-compatible with the existing service APIs, so there is no need for you to update your current client code or applications. We will keep clients of OpenSearch compatible with open source.

To get started, in the AWS Management Console, choose Create a domain. In Step 1: Choose deployment type, select OpenSearch 1.0 (latest).

2021 opensearch service 2

We recommend OpenSearch 1.0 if you are deploying a new cluster and want access to the latest features and enhancements. OpenSearch 1.0 is compatible with the open-source Elasticsearch 7.10 APIs and most clients.

Upgrading to OpenSearch 1.0
Amazon OpenSearch Service offers a seamless in-place upgrade path from existing Elasticsearch 6.x and 7.x managed clusters to OpenSearch. To upgrade a domain to OpenSearch 1.0 in the AWS Management Console, choose the domain that you want to upgrade, choose Actions, and then select Upgrade domain.

2021 opensearch service 4

Next, you can select a version to upgrade to OpenSearch 1.0 for your existing domain without creating a separate domain and migrating your data.

The upgrade process is irreversible. It can’t be paused or canceled. During an upgrade, you can’t make configuration changes to the domain. Before you start an upgrade, you can perform the pre-upgrade checks for issues that can block an upgrade and take a snapshot of the cluster by selecting Check upgrade eligibility.

2021 opensearch service 3

Amazon OpenSearch Services starts the upgrade, which can take from 15 minutes to several hours to complete. To learn more, see Upgrading Elasticsearch and Service Software Updates in Amazon OpenSearch Service Developer Guide.

OpenSearch Features
OpenSearch provides the following features that were not previously available in open-source Elasticsearch.

Advanced SecurityOffers encryption, authentication, authorization, and auditing features. They include integrations with Active Directory, LDAP, SAML, Kerberos, JSON web tokens, and more. OpenSearch also provides fine-grained, role-based access control to indices, documents, and fields.
SQL Query SyntaxProvides the familiar SQL query syntax. Use aggregations, group by, and where clauses to investigate your data. Read data as JSON documents or CSV tables so you have the flexibility to use the format that works best for you.
ReportingSchedule, export, and share reports from dashboards, saved searches, alerts, and visualizations.
Anomaly DetectionUse machine learning anomaly detection based on the Random Cut Forest (RCF) algorithm to automatically detect anomalies as your data is ingested. Combine with alerting to monitor data in near real time and send alert notifications automatically.
Index ManagementDefine custom policies to automate routine index management tasks, such as rollover and delete, apply them to indices and index patterns, and transforms.
Performance Analyzer and RCA FrameworkQuery numerous cluster performance metrics and aggregations. Use PerfTop, the command line interface (CLI) to quickly display and analyze those metrics. Use the root cause analysis (RCA) framework to investigate performance and reliability issues in clusters.
Asynchronous SearchRun complex queries without worrying about the query timing out with Asynchronous Search queries running in the background. Track query progress and retrieve partial results as they become available.
Trace AnalyticsIngest and visualize OpenTelemetry data for distributed applications. Visualize the flow of events between these applications to identify performance problems.
AlertingAutomatically monitor data and send alert notifications to stakeholders. With an intuitive interface and a powerful API, easily set up, manage, and monitor alerts. Craft highly specific alert conditions using OpenSearch’s full query language and scripting capabilities.
k-NN searchUsing machine learning, run the nearest neighbor search algorithm on billions of documents across thousands of dimensions with the same ease as running any regular OpenSearch query. Use aggregations and filter clauses to further refine similarity search operations. k-NN similarity search powers use cases such as product recommendations, fraud detection, image and video search, related document search, and more.
Piped Processing LanguageProvides a familiar query syntax with a comprehensive set of commands delimited by pipes (|) to query data.
Dashboard NotebooksCombine dashboards, visualizations, text, and more to provide context and detailed explanations when analyzing data.

OpenSearch 1.0 supports three new features that are not available in the existing Elasticsearch versions supported on Amazon OpenSearch Service: Transforms, Data Streams, and Notebooks in OpenSearch Dashboards.

To engage with the OpenSearch community, we welcome pull requests through GitHub to fix bugs, improve performance and stability, or add new features. You can leave feedback in the OpenSearch community forum.

Now Available
Starting today, Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service in all AWS Regions. For more information, see the Amazon OpenSearch Service page.

You can send feedback to the AWS forum for Amazon OpenSearch Service or through your usual AWS Support contacts.


AmazonWebServicesBlog?d=yIl2AUoC8zA AmazonWebServicesBlog?d=dnMXMwOfBR0 AmazonWebServicesBlog?d=7Q72WNTAKBA

Categories: AWS