FAQ

The following are frequently asked questions (FAQs) about DataStream 2.

DataStream FAQs
Question Answer
What is DataStream’s output data format? The output log file format for DataStream is Structured (CSV) or JSON. For sample schemas and data sets, see DataStream 2 API.
How many endpoints does DataStream send logs from? DataStream sends logs to customer destinations from multiple source endpoints. Typically, the number of source endpoints range from 10s to 100s and can change without prior notice.
What happens if log files fail to upload to a destination? Streaming logs to third-party destinations may temporarily fail due to issues such as latency or connection problems. To prevent data loss during temporary upload failures, DataStream retries data upload three times. After three failed upload retries, the data will not be saved.

You can configure the Alerts feature to get notified about upload failures.

Should I use log data for billing? We recommend using raw log data for basic traffic analysis and monitoring CDN health. In case of issues with third-party destinations, such as latency or connection problems, data will be lost after three failed retries to connect.

You should take these limitations into account before using data served on your stream for audit, compliance and billing purposes. See DataStream use cases.

Does DataStream store log data in case of stream failures? DataStream doesn't store data streamed to third-party destinations. In case of stream failure, the stream attempts to reconnect three times every 5 minutes. After three failures over 15 minutes, the log data is lost.
I don’t use a log analytics tool. Can I still use DataStream? You need a tool to parse and visualize the DataStream output. You could choose lower cost, open source stacks for data parsing and visualization in human readable formats.
Does DataStream support security event logs? DataStream is a log delivery product for all transactional events and associated metrics. You can use the SIEM Integration product to deliver security logs. See SIEM Integration.
Why does one or more custom fields return ^ in log files? DataStream requires enabling custom fields, such as User-Agent, Accept-Language, Cookie, Referrer or X-Forwarded-For in the Log Delivery Service behavior in your property configuration. If log lines return ^, first enable the fields in Property Manager. See Enable logging custom parameters.