why a developer writes

Read D3.js in Action

| Comments

Recently at work, I have been working on developing a few features on top of Apache Airflow. Some of the features are UI heavy, and require some amount of the data visualization using D3.js. While working on those features, I thought it could be a good chance to spend some time on learning D3.js in-depth, so I chose to read this book D3.js in Action on Here comes a summary of this book, and some notes I took while reading.

Overall I find this book is easy to read as long as you have some knowledge in JavaScript. In this book, a few key concepts in D3.js are clearly laid out, and the examples cover a good set of common usages and tactics you need to know for building data visualization features using D3.js.

This book has 11 chapters. Chapter 1, 2 and 3 are introduction to D3.js, the high level flow and common operations of using D3.js for information visualization, and how to structure a data visualization project with D3.js.

In the first three chapters, alongside with the basic concepts, a few tactics I find worth learning from the very beginning are:

  • Integrate scales in data binding: D3.js provides handy scale functions to normalize data values for better display. Example built-in scale functions include: d3.scaleLinear(), d3.scaleSequential(), d3.scaleQuantize() and so on. A D3 scale has two primary functions: ..domain() and .range(), both of which expect arrays and must have arrays of the same length to get the right results. The array in .domain() indicates the series of values being mapped to .range().

  • Enter, update, merge, and exit to update DOM elements: Understanding how to create, change, and move elements using enter(), exit(), and selections is the basis for all the complex D3 functionality. One note here is D3 doesn’t follow the convention that when the data changes, the corresponding display is updated; you need to build that functionality yourself.

  • Getting access to the actual DOM element in the selection can be accomplished in one of two ways:

    • Using this in the inline functions (cannot be used with arrow functions)
    • Using the .node() function

Using this

3"circle").each(function(d,i) {

Using .node() function


Chapter 4, 5, 6, 7 and 8 introduce the methods and details of building specific types of visualization for specific types of data: chart components, layouts, complex hierarchical data visualization, network visualization, and visualizing geospatial information.

  • One note about Layouts: D3 contains a variety of functions, referred to as layouts, that help you format your data so that it can be presented using a popular charting method. D3 layouts don’t result in charts; they result in the settings necessary for charts. Example D3 built-in layouts: d3.layout.histogram(), d3.layout.pie(), d3.layout.tree() etc.

Chapter 9 covers how to using D3 with React. The challenge of integrating D3 with React is that React and D3 both want to control the DOM. The entire select/enter/exit/update pattern with D3 is in direct conflict with React and its virtual DOM. The way most people use D3 with React is to use React to build the structure of the application, and to render traditional HTML elements, and then when it comes to the data visualization section, they pass a DOM container (typically an <svg>) over to D3 and use D3 to create and destroy and update elements.

Chapter 10 and 11 are advanced usages about customizing layouts and components, and mixed mode rendering in HTML canvas.

Learn Golang - a Mind Map

| Comments

Earlier this year I started to learn Golang. There were three good reasons why I wanted to learn this programming language.

  1. For work: Some of the projects my team have worked on, or planned to work on, use Golang to improve applications’ performance.
  2. For better understanding cloud infrastructure: Some of the key cloud infrastructure open source projects, including Kubernetes and Docker, are written in Golang.
  3. For using it to easily create multithreaded and concurrent programs.

One thing I did when learning Golang was creating a mind mapping diagram. The mind map helps me to organize the different topics of Golang that I need to learn about, and to dig into each part without getting lost in too many details. It also makes it much easier to remember information.

If you are also learning Golang, you can take a look at the Golang mind map here on my github. It mainly covers Golang syntax, flow control, data structures, methods, functions, interfaces and basic concurrency. One thing it doesn’t have yet is Go Modules, which was added in Go 1.11 (released in August 2018). As Go dev team announced, current module support is priliminary. In Go 1.12, scheduled for February 2019, they will refine module support. I will update this mind map to add Go Modules then.

Lastly, 2019 is around the corner. Happy New Year!

Reading Notes - Designing Distributed Systems

| Comments

Recently I read a book Designing Distributed Systems, which is written by Brendan Burns, and published by O’Reilly earlier this year. This book introduces the patterns and components used in the development of distributed systems.

Before I started to read this book, I had three questions in my mind, and try to find the answers from the book. Those three questions are:

  1. What’s the most important difference between designing distributed systems and single machine systems?
  2. Why container technology, such as docker, kubernates, is so popular? How could it be helpful?
  3. What are the common patterns used in distributed systems design, and when shall I use them?

This book does give me the answers, at least partial ones. I put my reading notes into a Google Slides, and you can find it here to read the details. A PDF version in light background color is available here.

The short answers to my questions are as in the following:

Performance Profiling Tools on Windows

| Comments

Last year, I wrote a blog post about CPU Profiling and the tools on Linux. Today I’m going to write about a few Performance profiling tools on Windows platform. Last week I was working on profiling and analyzing the build process of a project on Windows, and during the process I experimented with a few different tools.

Performance Monitor (perfmon)

Performance Monitor is a small utility provided by Windows OS, you can start it by running command perfmon. With perfmon, you can monitor real-time system performance, and record performance to files for post analysis. This tool provides some extremely useful interfaces in its GUI.

Real-time Performance

To view current performance activity, you just need click on the Performance Monitor button in the left panel:

Effective Jenkins Plugins (1)

| Comments

(Update 2018-05-13: Here is a translation of this post in Hindi, provided by Nikol @ DealsDaddy.)

Jenkins, originally founded in 2006 as “Hudson”, is one of the leading automation applications which support building, deploying and automating software projects. One great advantage of Jenkins is there are hundreds of plugins available which enable various kinds of extended features needed in the Continous Integration and Continuous Delivery process. As I just checked on the Jenkins Plugins page, there are 873 plugins that fall into five categories: Platforms, User interface, Administration, Source code management and Build management.

Effectively using Jenkins plugins makes your experience with Jenkins more productive. I’m going to occasionally write about Jenkins plugins that I used or learned about. The first post will start from some of the plugins I used when I worked on building a Continuous Delivery system last year (from 2015 to 2016).

Job Configuration History

JobConfigHistory Plugin

This plugin saves every change made to a job. It allows you to see the history of job configurations, compare configuration differences, and restore a particular version of config. You can also see which user makes a particular change if you configured a security policy.