Frequently asked questions

The following are frequently asked questions (FAQs) about DataStream.

DataStream FAQs
Question Answer
What is DataStream’s output data format? The output format for DataStream is JSON. For sample schemas and data sets, see DataStream API.
When would I use raw logs as opposed to aggregated metrics? You can start with aggregated metrics for a snapshot of CDN health. If you see any anomalies, you can start streaming raw logs for root cause analysis or diagnostics. For example, if you’re experiencing a large number of 4xx errors over a period of time, raw logs can help you find the cause and potential fixes.

You can turn raw log streams on before and after a new deployment and turn them off once the deployment is stable. For example, before and after offload monitoring. This helps avoid billing when you don’t need the logs.

Which connectors does DataStream currently support? DataStream comes with default out-of-the-box push-based connectors for Splunk, Sumo Logic, and Amazon S3. See Key concepts and terms.
I don’t use a log analytics tool. Can I still use DataStream’s raw logs or aggregated metrics? You need a tool to parse and visualize the DataStream output. You could choose lower cost, open source stacks for data parsing and visualization in human readable formats.
How does DataStream aggregation work? DataStream aggregation is based on a tumbling window over the selected aggregation time frame. Tumbling windows are a series of fixed-sized, non-overlapping, and contiguous time intervals.

DataStream aggregation operators start collecting live data based on the aggregation time frame chosen at the time of defining a stream. For example, if you create a stream at 10.00 with a five-minute time frame, DataStream will collect and aggregate data from 10.00 to 10.05. DataStream aggregation operators will repeat the aggregation every five minutes.

What are the minimum and maximum aggregation time frames available? These are the available time frames:
  • 1 minute
  • 5 minutes
  • 15 minutes
  • 30 minutes
  • 1 hour
Note: The aggregated data is retained for 12 hours on a rolling window basis.
How is DataStream different from the CloudMonitor or LDS products that I already use? There are new features that make DataStream the next generation log delivery product compared to LDS and CloudMonitor.
  • Data retention
  • Data aggregation
  • Pull APIs in addition to the traditional Push mechanism
  • Lower latency than CloudMonitor or LDS
  • Ability to define a stream with only the chosen data sets
  • Ability to turn a stream on and off as needed
Does DataStream support security event logs? DataStream is a log delivery product for all transactional events and associated metrics. You can use the SIEM Integration product to deliver security logs. See SIEM Integration.