Have you ever seen
In this article, I present a case study of adopting the background job and queue model to a real system. Given the current system architecture, an improved version is proposed and the details of designing such the system from different aspects will be provided. If the whole content did not make much sense, I highly recommend reading my other articles in the following order:
A notification system written in Python and MySQL has several performance issues, that are
This post aims to provide you an overview of how to fulfill requirements from the perspective of system design. Having said that, its content is not technology-driven, meaning that the implementation of what discussed is transparent to something like Kafka or whatever message queues like RabitMQ. There are no such features specific to a certain system or technology or framework.
In my previous articles (here and here), I did not discuss the relationship between jobs, assuming that they are non-related. By doing so, I simplified things, can do any scale up/down/out on the Worker Nodes as many as you want…
In my previous article, I introduce the concepts of queue and background jobs via an illustrative pharmacy example, its application in practice together with advantages and limitations. This article is part two of it, reviewing different job types and how it comes to the picture via several common applications.
There are a couple of background jobs, depending on how and on which kind of data they perform. I suggest you find the details in [1]. Here I have summarized some important aspects and practical use cases that you can imagine how background jobs look like.
In general, background jobs are…
Just a kind note: Since Medium does not support maths equations, many of them in this post were changed to raw representation. In case you dislike reading them (as I do), you can find better content here.
Previously, we introduced collaborative filtering (CF) recommendation system based on the behaviors of users or items in the neighborhood. At this time, another approach to build a CF system is presented — via solving the matrix factorization (MF) or matrix decomposition problem. It is called matrix factorization collaborative filtering (MFCF).
Recall that for content-based recommendation systems, each item is represented by a vector…
Just a kind note: Since Medium does not support maths formula, many of them in this post were changed to raw representation. In case you dislike reading them (as I), you can find better content here.
I introduced Content-based Recommendation Systems in the last article, which aims to provide item recommendations to the users based on the item’s features. Basically, the content-based approach builds a model for every user interest regardless of the presence of other users but items’ profiles. The advantage of this approach lies in its requirement of memory resources and computation time. It leverages the features of…
As a netizen, probably you have met the following facts many times:
There are actually many examples of systems…
In this article, I introduce basic concepts of a Recommendation System (RS) and 2 basic approaches of building an RS, namely Content-based and Collaborating Filtering. This time we dig into the implementation of a simple RS using Collaborating Filter approach using the item-based technique. You can The user-based one is
As mentioned in my previous article, CF is built based on the assumption of shared interests among different individuals. You may think about the law of large numbers and theories of crowd behavior. In a nutshell, it is stated as follows:
If 2 users u1 and u2 review n similar…
Here is what happens at the pharmacy store when an order comes:
Then another guy comes, the pharmacist repeats that process. And another 3 guys come, he still does that one by one. But when 5–10 guys come which might happen a couple of times during a day, he is definitely screwed up.
And if you told him something like “dude, why don’t you just get 1…
This post is intentionally for an overview of modeling network traffic, which I read from this article. I think it is useful for my later work which is potentially related to analyzing, evaluating, or simulating network flows.
There are two common representations for modeling network traffic [1].
In case of compound traffic of which the arrivals may happen in batches (several arrivals happen at the same moment), an additional sequence (which element is…
PhD student, Senior Software Developer