Using Kafka in Python for Simultaneous Data

Here's an example code in Python that demonstrates how to use Kafka to produce and consume data at the same time: ```python from kafka import KafkaProducer, KafkaConsumer import threading import time # Kafka Producer def produce_data(): producer = KafkaProducer(bootstrap_servers='localhost:9092') while True: message = "Hello, Kafka!" producer.send('my-topic', message.encode('utf-8')) print(f"Produced: {message}") time.sleep(2) # Kafka Consumer def consume_data(): consumer = KafkaConsumer('my-topic', bootstrap_servers='localhost:9092') for message in consumer: print(f"Consumed: {message.value.decode('utf-8')}") # Run producer and consumer in parallel producer_thread = threading.Thread(target=produce_data) consumer_thread = threading.Thread(target=consume_data) producer_thread.start() consumer_thread.start() producer_thread.join() consumer_thread.join() ``` Here's how the code works: 1. The `produce_data()` function creates a Kafka producer, and in a loop, it sends a message to the 'my-topic' topic every 2 seconds. 2. The `consume_data()` function creates a Kafka consumer, and in a loop, it reads and prints the messages from the 'my-topic' topic. 3. The `producer_thread` and `consumer_thread` are created and started in parallel using the `threading` module. 4. The `join()` method is called on both threads to wait for them to finish. When you run this code, you should see the producer sending messages and the consumer consuming them at the same time. Note that you need to have a Kafka server running on your local machine (or a remote server) for this code to work. You can download and install Kafka from the official website: https://kafka.apache.org/downloads Also, make sure to install the `kafka-python` library using `pip install kafka-python` before running the code.

Write a code to use kafka in python to produce and consume data at same time

6 months ago