Posts

Showing posts with the label Data-storage

Featured Post

Python Program: JSON to CSV Conversion

-
Image
JavaScript object notion is also called JSON file, it's data you can write to a CSV file. Here's a sample python logic for your ready reference.  You can write a simple python program by importing the JSON, and CSV packages. This is your first step. It is helpful to use all the JSON methods in your python logic. That means the required package is JSON. So far, so good. In the next step, I'll show you how to write a Python program. You'll also find each term explained. What is JSON File JSON is key value pair file. The popular use of JSON file is to transmit data between heterogeneous applications. Python supports JSON file. What is CSV File The CSV is comma separated file. It is popularly used to send and receive data. How to Write JSON file data to a CSV file Here the JSON data that has written to CSV file. It's simple method and you can use for CSV file conversion use. import csv, json json_string = '[{"value1": 1, "value2": 2,"value3
Post a Comment
Read more

How to Retain data in Kafka and Get Additional Time for Analysis

-
Image
The default topic retention period in Kafka is seven days. However, you can change the current retention period and keep data for a few more days. Hence it provides you additional time for analysis to get business insights. Kafka retention period The retention period, you can set on two parameters of bytes and time. Due to cheap storage costs, companies wish to extend the data retention period.  The retention period setup you need in the broker.  It is not a deviation that Kafka is designed only for Seven days, and why we need to change it. Since space is cheaper, we can extend the retention period. Setup for the retention period Below is the setup in the broker configuration file for the retention period. log.retention.bytes The most significant size threshold in bytes for deleting a log. log.retention.ms The length in milliseconds of a log will be maintained before being deleted. log.retention.minutes Length before deletion in minutes. log.retention.ms is used as well if both are set
Post a Comment
Read more