Key concepts and terms
Here’s a brief overview of key concepts in the DataStream application.
- Lets you create data streams for your properties that provide low latency streaming of data in raw and aggregated forms. You can configure your data stream to either send log data to one or more destinations or integrate it with third-party web services or your own solution and access the data through the DataStream pull API.
- Collects logs about edge request-response cycles for multiple properties. Based on the type of destination, it either streams raw data or aggregated metrics to a destination or stores the data in DataStream buffer and makes it available through the DataStream pull API
- Types of logs
- You can configure your data stream to collect and deliver these types of
- Raw data
- Raw log information about individual edge request-response cycles on the Akamai platform. Depending on your stream's destination type, your stream sends bundled JSON logs to destinations or makes these logs available through the DataStream pull API. You can use raw data logs to find details about specific incidents, search the logs for instances using a specific IP address, or analyze the patterns of multiple attacks.
- Aggregated metrics
- Information about edge request-response cycles aggregated over selected metrics and a time window. Depending on your stream's destination type, your stream sends aggregated JSON metrics to destinations or makes these metrics available through the DataStream pull API. You can use aggregated logs to search for the root cause of an error and monitor the performance, security status, and general behavior of an application.
- Data set
- A predefined set of parameters that you can monitor in edge request-response cycles or metrics that you use as categories for aggregation. You can choose or ignore which parameters you want to receive in your logs or metrics over which you want to aggregate these logs. See Data set parameters.
- The destination where a data stream sends or stores log data. You can
configure multiple destinations for a data stream. These are possible destinations:
- Push destination
- Defines a destination that sends log data to a specific destination
for storing, monitoring, and analytical purposes.
- Amazon S3
- Provides cloud object storage for your data. See Getting started with Amazon S3.
- Provides an interface that lets you search, monitor, and analyze your data. See Splunk solutions.
- Sumo Logic
- Provides a cloud based service that lets you easily collect, centralize, and integrate data sources. See How it works.
- Provides an ability to send your data to a secure URL of your choosing. Make sure that the URL handles user authentication.
- Provides an interface that lets you monitor cloud applications through analyzing data. See Datadog documentation.
- Pull destination
- Defines a destination that doesn't send log data to any specific
destination but stores it in a DataStream buffer and makes it available to your
integrations through the DataStream pull API for up to 12 hours. See Integration.
- DataStream buffer
- Provides storage of log data for up to 12 hours. It lets you access your log data with the DataStream pull API. Note that you can configure only one DataStream buffer for a stream.
- A web service that you want to integrate with a data stream and pull your
data for analytical purposes. You can only configure an integration for a stream that uses
a DataStream buffer as a destination. These are some of the possible integrations:
- Provides data analysis without having to take care of the underlying infrastructure. It also lets you visualize your data with an integrated tool called Data Monitor. See BigQuery integration.
- Provides out-of-the-box integration with DataStream to measure the health of your application performance in real-time. See Datadog integration.
- Content delivery property
- You manage content delivery by creating and configuring delivery properties.
These configurations control how edge servers handle and respond to HTTP(S) requests from
end users. For example, requests for web sites, software or game downloads, audio and
video streams, or images. See Content delivery products.
DataStream offers the ability to collect performance logs against selected delivery properties and streams them to configured destinations. This facilitates downstream data reporting, analytics for usage, and delivery performance visibility.DataStream lets you:
- monitor up to 15 properties in a data stream.
- monitor logs for the same property in multiple data streams.
- enable log collection for all requests within a property.
- enable log collection for requests matching specific criteria.
- Pull API
- A reporting API that lets you access log data collected by your data streams if you store it in a DataStream buffer. This API can fetch log data for up to 12 hours. See DataStream pull API.