Truncate your time series data!
There is some cases we required to truncate our data. For example, count how many times A and B occurred last day.…
Create real-time streaming data (WebSocket) using Tornado
Today learnt tornado to serve stream data to user. Tornado is python web framework, but it is interesting because it only runs a thread per process but still can serve many clients due to its asynchronous design. Wait, asynchronous? Actually there are many terminologies you need to know. Concurrency, asynchronous,…
Fixing Kafka STRING key and AVRO value Python
confluent_kafka.avro.serializer.SerializerError: message does not start with magic byte Damn confluent-python... I faced a lot of challenges when trying to set up kafka thing. First, I encountered listener protocol mapping, then KSQL, and now even confluent-kafka. -.- The story began when I tried to consume a…
Simple Neural Network using Tensorflow Part 1
WHATNeural network is similar like human neurons. They connect with each other. It consists of 3 type of layers: input, hidden, output. If you fit data into a neural network, it will process and produce a good (or bad) prediction. Normally, it performs two phrases: forward and back-propagation. Today we…
How to Create Kafka using Docker in 5 min
This experiment I spent so much time on it finding the source and figuring out how Kafka works. Damnn... I guess everyone already knew what Kafka is, so let's get into the installation step. Before that, there are some requirements needed before installing Kafka. This is just only development purposes.…