How did I develop a crypto service for Tableau Server

For more information about Tableau Server, you can read my blogs here

What is this project about?

Tableau Server is an on-premise data analytics and visualization platform. Customer can deploy bits in their own data center, or in AWS, Azure, and GCP.

There are two categories of passwords within a Tableau Server cluster:

  • User input passwords, which are entered by users
  • AWS key
  • smtp passwords
  • Windows runas user password
  • Server managed passwords, which are randomly generated by server

  • jdbc password
  • Postgres admin password
  • ZooKeeper password

So the passwords I’m talking about is not the password for someone’s server account. The latter ones live in a table of the cluster’s PostgreSQL database and are already encrypted.

The current version of Tableau Server (v10) stores all the passwords in plain text in a configuration file, which is distributed among machines.

The new version of Tableau Server also stores all passwords in plain text, but in a Znode in ZooKeeper, as I described in this blog.

The project is about developing a crypto service that encrypts passwords before persisting them somewhere, and decrypts passwords into memory when server uses them, so that no more clear text secrets live in Tableau Server. The new service should support both our old monolith JRuby app and our new service-oriented server architecture.

Continue reading “How did I develop a crypto service for Tableau Server”

How did I build a custom dynamic ZooKeeper management and migration framework and an asynchronous job framework for Tableau Server 11


First of all, Tableau Server 11 is not out yet. Tableau Server 10 is just released yesterday.

Product Strategy Overview

Tableau Server is an enterpise data analytics and visualization platform, and our customers are business people. In order to make the project usage experience as smooth as possible, we believe we should enable business people to manage Tableau Server on their own without any help from enterprise IT.

Therefore, one of the most critical goals for developing Tableau Server is that Tableau will provide GUIs and built-in workflows to automate as many managing processes as possible.

Project and Feature Requirements

Tableau core server team is building a new Java-based Tableau Server architecutre for Tableau Server 11 (target release, might change in the future) to replace the old jruby one.

Notably, old jruby Tabadmin persisting cluster-wide configuration and topology information in a local file (workgroup.yml). The role of ZooKeeper (referenced as ZK) is only used a distributed lock and a temp data store – it keeps some vilatile data that can be lost or wiped out and no one cares about it. This characteristic also makes managing and migrating ZooKeeper very very easy – you only need to clean up all data in existing ZooKeeper nodes, deploy the ZooKeeper bits to new nodes, and bring the new cluster up.

Unlike the old world, new Tabadmin will persist all essential configuration and topology data in ZK as a single source of truth, which makes managing, expanding, and migrating ZK much harder than before.

The Problem

The problem we are facing is how to maintain ZK’s data and expand/shrink ZK cluster, as well as making this process super reliable and easy for users.

Let’s elaborate more about the problem.

Say we start with a single Tableau Server node A, which has a single node ZK cluster runs on it. Then we want to expand the Tableau Server cluster from 1 node (A) to 3 nodes (A, B, C), as well as the ZK cluster runs on top of it. How to achieve that?

(A) -> (A, B, C)


For customers who already have a ZK cluster in house and want to reuse that cluster, Tableau Server provides configurations that enable them to do so. That is not part of the project, and will not be discussed here. This project is focusing on smoothing the experience of managing Tableau Server’s built-in ZooKeeper cluster.

Naive Approach

The naive approach is to shut down (A), deploy ZK bits on B and C, change their configuration files, and bring up (A, B, C). Well, this can lead to disasters. The reason being that ZK runs on a quorum mode, means that if more than half nodes in a ZK cluster form a quorum, their decision will be the final decision that all nodes should follow. In this case, if B and C come up first and form a quorum, their decision will be “there’s no data in our cluster”. When A joins them, B and C will force A to delete its data, causing data loss.

ZooKeeper’s built-in Approach

Actually all developers using ZooKeeper faces the same problem. Think about running a ZooKeeper in your data center. If a ZooKeeper node fails, the common practice is that admin will try to bring it back manually/through scripts. If the machine dies, admins have to bring up a new machine, change its IP to match the failed one, and join it to the cluster. And you must know that machine failure is more than normal.

So ZooKeeper 3.5 starts to build a native rolling based approach to solve this common problem. The principle is to bring up a new node each time, join it to the existing cluster, wait them to sync up and become stable, bring up another new node.

(A) -> (A, B) -> (A, B, C)

The problem is that ZK is not originally developed to support this use case. It involves lots of refactoring, produces lots of bugs, and is not stable for production.

Take a look at ZK’s release page. 3.5.0 was released around 3 years ago. Apache then release 3.5.1 and 3.5.2, both are alpha version. They themselves even don’t recommend using it for production.


Continue reading “How did I build a custom dynamic ZooKeeper management and migration framework and an asynchronous job framework for Tableau Server 11”

How did I build a Facebook Messenger chatbot for Tableau’s ‘VizOfTheDay’ feed

What is Viz Of The Day

It’s a Tableau Public product used mainly for promoting Tableau as a brand.

What is Facebook Messenger chatbot

What can my bot do?

  • Users can query VizOfTheDay on any date through either a hardcoded format, or natural language (which will be processed by If the bot doesn’t find that or cannot recognize the message, it will return a random one for users.
  • Users can subscribe to VizOfTheDay, so that when a new viz is published on Tableau Public, users will be notified on Messenger App by a push notification. Of course, they can also unsubscribe it.

How does the system look like

  • Backend running on a single AWS EC2
  • ELB as gateway, to bind a domain name for https
  • Using DynamoDB for storage
  • Calling Facebook Messenger API to create UI
  • Taking advantages of for some natural language processing

Modules and their Responsibilities:

  • FeedProcessor
  • periodically retrieves VizOfTheDay from a Tableau Public endpoint
  • serves queries of viz on a specified date (can be today or any date before)
  • Controller
  • receives, processes, and interprets requests from Facebook Messenger
  • distributes interpretted requests to corresponding services
  • Response Service
  • Takes responsibilities of replying requests
  • Supports different kind of templates, from pure text, pure thumbnails, to a combination of text thumbnails, and multiple tabs
  • User Service
  • Stores and serves queries on users’ meta data (id, first and last name, etc) and subscription options (subscribed or not)
  • Data is cached in memory for faster access to properties like first name and subscription option
  • Data is persisted in DynamoDB
  • Profile Querying Service
  • Takes a user id and query its meta data against Facebook Graph API
  • Push Notifier Service
  • Runs daily
  • Once detects a new viz from FeedProcessor, push it to all subscribed users by getting users from User Service
Facebook Messenger API
|       for interpretation
Controller -----------
|                                         |
|                                         |
enqueue id | |
| |
| |
DynamoDB -- User Service ------ Response Service
| | |
| | |
Profile Querying Serivce | |
| |
Push Notifier |
| |
| |
| |
Feed Processor----------

Continue reading “How did I build a Facebook Messenger chatbot for Tableau’s ‘VizOfTheDay’ feed”