The following are frequently asked questions (FAQs) about DataStream.
|What is DataStream’s output data format?||The output format for DataStream is JSON. For sample schemas and data sets, see DataStream API.|
|What are the usage limits?||For DataStream, the rate limit on the free tier is 380
requests per second, which can add up to one billion request or 5000GB
per month. If the actual traffic is higher, the system samples the data
to ensure the limit of 380 requests per second.
DataStream offers a paid tier for customers who need data delivery in excess of 5000GB or 1 billion requests per month.
|How many endpoints does DataStream send logs from?||DataStream sends logs to customer destinations from multiple source endpoints. Typically, the number of source endpoints range from 10s to 100s and can change without prior notice.|
|When would I use raw logs as opposed to aggregated metrics?||You can start with aggregated metrics for a snapshot of CDN health.
If you see any anomalies, you can start streaming raw logs for root
cause analysis or diagnostics. For example, if you’re experiencing a
large number of 4xx errors over a period of time, raw logs can help you
find the cause and potential fixes.
You can turn raw log streams on before and after a new deployment and turn them off once the deployment is stable. For example, before and after offload monitoring. This helps avoid billing when you don’t need the logs.
|I don’t use a log analytics tool. Can I still use DataStream’s raw logs or aggregated metrics?||You need a tool to parse and visualize the DataStream output. You could choose lower cost, open source stacks for data parsing and visualization in human readable formats.|
|How does DataStream aggregation work?||DataStream aggregation is based on a tumbling window over the
selected aggregation time frame. Tumbling windows are a series of
fixed-sized, non-overlapping, and contiguous time
DataStream aggregation operators start collecting live data based on the aggregation time frame chosen at the time of defining a stream. For example, if you create a stream at 10.00 with a five-minute time frame, DataStream will collect and aggregate data from 10.00 to 10.05. DataStream aggregation operators will repeat the aggregation every five minutes.
|What are the minimum and maximum aggregation time frames available?||These are the available time frames:
Note: The aggregated data is retained for 12 hours on a rolling window basis.
|How is DataStream different from the CloudMonitor or LDS products that I already use?||There are new features that make DataStream the next generation log
delivery product compared to LDS and CloudMonitor.
|Does DataStream support security event logs?||DataStream is a log delivery product for all transactional events and associated metrics. You can use the SIEM Integration product to deliver security logs. See SIEM Integration.|
|Why does one or more custom fields return
||DataStream requires enabling custom fields, such as User-Agent,
Accept-Language, Cookie, Referrer or X-Forwarded-For in the Log Delivery
Service behavior in your property configuration. If log lines return