CODEX

Have you ever seen

  • Your parents just start using Facebook and make friends with some. After a couple of days, Facebook suggests some far far far relatives whom you even have no idea.
  • You are doing window shopping and hope to find stuff that you want to buy. Then a bunch of items like jeans, handbags, belts, etc. After a while, do you realize how far you have gone following those things and totally forgot your original purpose?
  • You are not in the mood and listening to several pop songs. …

CODEX

In this article, I present a case study of adopting the background job and queue model to a real system. Given the current system architecture, an improved version is proposed and the details of designing such the system from different aspects will be provided. If the whole content did not make much sense, I highly recommend reading my other articles in the following order:

Input

A notification system written in Python and MySQL has several performance issues, that are

  • Latency that slows down services
  • High service demand during…

Photo by Cesar Carlevarino Aragon on Unsplash

CODEX

This post aims to provide you an overview of how to fulfill requirements from the perspective of system design. Having said that, its content is not technology-driven, meaning that the implementation of what discussed is transparent to something like Kafka or whatever message queues like RabitMQ. There are no such features specific to a certain system or technology or framework.

In my previous articles (here and here), I did not discuss the relationship between jobs, assuming that they are non-related. By doing so, I simplified things, can do any scale up/down/out on the Worker Nodes as many as you want…


In my previous article, I introduce the concepts of queue and background jobs via an illustrative pharmacy example, its application in practice together with advantages and limitations. This article is part two of it, reviewing different job types and how it comes to the picture via several common applications.

Job Definition and Categories

There are a couple of background jobs, depending on how and on which kind of data they perform. I suggest you find the details in [1]. Here I have summarized some important aspects and practical use cases that you can imagine how background jobs look like.

In general, background jobs are…


Just a kind note: Since Medium does not support maths equations, many of them in this post were changed to raw representation. In case you dislike reading them (as I do), you can find better content here.

Introduction

Previously, we introduced collaborative filtering (CF) recommendation system based on the behaviors of users or items in the neighborhood. At this time, another approach to build a CF system is presented — via solving the matrix factorization (MF) or matrix decomposition problem. It is called matrix factorization collaborative filtering (MFCF).

Recall that for content-based recommendation systems, each item is represented by a vector…


Just a kind note: Since Medium does not support maths formula, many of them in this post were changed to raw representation. In case you dislike reading them (as I), you can find better content here.

https://techsharing21.com/
https://techsharing21.com/

Introduction

I introduced Content-based Recommendation Systems in the last article, which aims to provide item recommendations to the users based on the item’s features. Basically, the content-based approach builds a model for every user interest regardless of the presence of other users but items’ profiles. The advantage of this approach lies in its requirement of memory resources and computation time. It leverages the features of…


Introduction

As a netizen, probably you have met the following facts many times:

  • Youtube automatically plays clips related to the one that you are watching. And especially when you are at its homepage, Youtube also automatically suggests clips that you find interesting.
  • Have you ever noticed the section [Frequently bought together] while doing shopping on Amazon?
  • Have you ever wondered that Facebook is secretly listening to your conversation and display the ads relevant to your favorite items even though they are only in your mind? Or friend suggestions?
  • Netflix suggests movies to its users.

There are actually many examples of systems…


Photo by Markus Spiske on Unsplash

CODEX

In this article, I introduce basic concepts of a Recommendation System (RS) and 2 basic approaches of building an RS, namely Content-based and Collaborating Filtering. This time we dig into the implementation of a simple RS using Collaborating Filter approach using the item-based technique. You can The user-based one is

Collaborating Filtering (CF)

As mentioned in my previous article, CF is built based on the assumption of shared interests among different individuals. You may think about the law of large numbers and theories of crowd behavior. In a nutshell, it is stated as follows:

If 2 users u1 and u2 review n similar…


At a pharmacy store

Here is what happens at the pharmacy store when an order comes:

  • The pharmacist receives your prescription and goes back to the store to get the prescribed medicine
  • Bring back to you and tell you how much
  • Payment and change (if any and if cash)

Then another guy comes, the pharmacist repeats that process. And another 3 guys come, he still does that one by one. But when 5–10 guys come which might happen a couple of times during a day, he is definitely screwed up.

And if you told him something like “dude, why don’t you just get 1…


Photo by Alina Grubnyak on Unsplash

CODEX

This post is intentionally for an overview of modeling network traffic, which I read from this article. I think it is useful for my later work which is potentially related to analyzing, evaluating, or simulating network flows.

There are two common representations for modeling network traffic [1].

  1. Counting process: how many packets arrive for a specific period of time
  2. Inter-arrival time process: how long did time pass between two successive arrivals (named as Aₙ)?

In case of compound traffic of which the arrivals may happen in batches (several arrivals happen at the same moment), an additional sequence (which element is…

Tuan Nguyen

PhD student, Senior Software Developer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store