Key concepts and terms

Here’s a brief overview of key concepts in the DataStream application.

DataStream
Lets you create data streams for your properties that provide low latency streaming of data in raw and aggregated forms. You can configure your data stream to either send log data to one or more destinations or integrate it with third-party web services or your own solution and access the data through the DataStream pull API.
Stream
Collects logs about edge request-response cycles for multiple properties. Based on the type of destination, it either streams raw data or aggregated metrics to a destination or stores the data in DataStream buffer and makes it available through the DataStream pull API
Types of logs
You can configure your data stream to collect and deliver these types of logs:
Raw data
Raw log information about individual edge request-response cycles on the Akamai platform. Depending on your stream's destination type, your stream sends bundled JSON logs to destinations or makes these logs available through the DataStream pull API. You can use raw data logs to find details about specific incidents, search the logs for instances using a specific IP address, or analyze the patterns of multiple attacks.
Aggregated metrics
Information about edge request-response cycles aggregated over selected metrics and a time window. Depending on your stream's destination type, your stream sends aggregated JSON metrics to destinations or makes these metrics available through the DataStream pull API. You can use aggregated logs to search for the root cause of an error and monitor the performance, security status, and general behavior of an application.
Data set
A predefined set of parameters that you can monitor in edge request-response cycles or metrics that you use as categories for aggregation. You can choose or ignore which parameters you want to receive in your logs or metrics over which you want to aggregate these logs. See Data set parameters.
Destination
The destination where a data stream sends or stores log data. You can configure multiple destinations for a data stream. These are possible destinations:
Push destination
Defines a destination that sends log data to a specific destination for storing, monitoring, and analytical purposes.
Amazon S3
Provides cloud object storage for your data. See Getting started with Amazon S3.
Splunk
Provides an interface that lets you search, monitor, and analyze your data. See Splunk solutions.
Sumo Logic
Provides a cloud based service that lets you easily collect, centralize, and integrate data sources. See How it works.
HTTPS
Provides an ability to send your data to a secure URL of your choosing. Make sure that the URL handles user authentication.
Datadog
Provides an interface that lets you monitor cloud applications through analyzing data. See Datadog documentation.
Pull destination
Defines a destination that doesn't send log data to any specific destination but stores it in a DataStream buffer and makes it available to your integrations through the DataStream pull API for up to 12 hours. See Integration.
DataStream buffer
Provides storage of log data for up to 12 hours. It lets you access your log data with the DataStream pull API. Note that you can configure only one DataStream buffer for a stream.
Integration
A web service that you want to integrate with a data stream and pull your data for analytical purposes. You can only configure an integration for a stream that uses a DataStream buffer as a destination. These are some of the possible integrations:
BigQuery
Provides data analysis without having to take care of the underlying infrastructure. It also lets you visualize your data with an integrated tool called Data Monitor. See BigQuery integration.
Datadog
Provides out-of-the-box integration with DataStream to measure the health of your application performance in real-time. See Datadog integration.
Content delivery property
You manage content delivery by creating and configuring delivery properties. These configurations control how edge servers handle and respond to HTTP(S) requests from end users. For example, requests for web sites, software or game downloads, audio and video streams, or images. See Content delivery products.

DataStream offers the ability to collect performance logs against selected delivery properties and streams them to configured destinations. This facilitates downstream data reporting, analytics for usage, and delivery performance visibility.

DataStream lets you:
  • monitor up to 15 properties in a data stream.
  • monitor logs for the same property in multiple data streams.
  • enable log collection for all requests within a property.
  • enable log collection for requests matching specific criteria.
Pull API
A reporting API that lets you access log data collected by your data streams if you store it in a DataStream buffer. This API can fetch log data for up to 12 hours. See DataStream pull API.