Kafka with Windows C and C#
Project: Kafka 0.8 working example for Windows C and C#
0) Download the following open source programs for C and C# Kafka clients
1) Deliver a working program where a Windows C program, another C program, and a C# program exchange messages using Kafka version 0.8
2) Compile programs in VS2012 and demonstrate that programs act as per specification (below)
3) Briefly document each function in-line (ie, in the code)
The project will use Kafka, version 0.8, to demonstrate Kafka's basic functionality.
The idea is to have all 3 programs send and receive messages using Kafka.
There are 3 programs in the project. These will be called:
CM [Client Manager] (written in C#)
Engine (written in C)
WorkerX (written in C)
-There will be 3 identical Worker programs, named Worker0, Worker1, Worker2
Each of the programs will have various input and output files,
CM_input.txt CM_sentToEngine.txt CM_receivedFromEngine.txt
Engine_receivedFromCM.txt Engine_sentToWorker.txt Engine_receivedFromWorker.txt Engine_sentToCM.txt
The sequence will be as follows:
Create a CM_input.txt file, with 20 messages, 1 per line.
Each message in the CM file should have the name of the program (CM), and the message number, from 0 to 19, corresponding to each line in the file. For example:
CM Message #0
CM Message #1
The CM will read these messages from its input file, and send them to the Engine, in sequential (line number) order.
It will also write each message sent to CM_sentToEngine.txt
The Engine will store the messages in a queue, and write each message to its Engine_receivedFromCM file, in the order received.
The Engine will then send the messages, in the order that they are received, to the the Worker programs. It will write each message
it sends to its Engine_sentToWorker.txt file.
The Engine should divide the messages equally between the Workers, as follows:
CM Message #0 will be sent to Worker0
CM Message #1 will be sent to Worker1
CM Message #2 will be sent to Worker2
However, the engine should not send the next message to the Worker until it has received the message back from the Worker. For example,
it will send messages 0, 1 and 2 to the corresponding workers. It will then send messages 3, 4, and 5, after the Worker sends back
the first message. The Engine should send the next message to each Worker as the Worker completes its task.
Each worker will write the message to its WorkerN_receivedFromEngine file. It will then appeand the words "sentBackFromWorkerN" to the message,
and write the message to its WorkerN_sentToEngine file. It will then sleep for n seconds, and send the modified message back to the Engine.
The sleep time should be: 5 seconds for Worker0, 15 seconds for Worker1, and 10 seconds for Worker2.
The Engine will write each message, in the order received from the Workers, to its Engine_receivedFromWorker file.
It will then send each message back to the CM program, in the order received.
The CM will write each message received to CM_receivedFromEngine.txt.