Plan to compare WordPress with Hexo on Github Pages

I’ve been blogging with Hexo configured on my Github Page for about 9 months, and my blog is phoenixjiangnan.github.io.

It’s actually quite exciting at first to configure and run your static blog website yourself. Well, as time passed, many problems have surfaced, among which the most significant one being expensive operation cost to maintain and configure my own Github Pages correctly.

Thus I decided to give WordPress a try, so I can focus more on writing itself.

I will try to move a few post to this platform first as a start. If it works, I’ll move all my post here, and will write a more detailed blog explaining the reason for the transition and the benefits.

See you soon!

How did I develop a crypto service for Tableau Server

For more information about Tableau Server, you can read my blogs here

What is this project about?

Tableau Server is an on-premise data analytics and visualization platform. Customer can deploy bits in their own data center, or in AWS, Azure, and GCP.

There are two categories of passwords within a Tableau Server cluster:

• User input passwords, which are entered by users
• AWS key
• smtp passwords
• Windows runas user password
• …

• Server managed passwords, which are randomly generated by server

• jdbc password
• Postgres admin password
• ZooKeeper password
• …

So the passwords I’m talking about is not the password for someone’s server account. The latter ones live in a table of the cluster’s PostgreSQL database and are already encrypted.

The current version of Tableau Server (v10) stores all the passwords in plain text in a configuration file, which is distributed among machines.

The new version of Tableau Server also stores all passwords in plain text, but in a Znode in ZooKeeper, as I described in this blog.

The project is about developing a crypto service that encrypts passwords before persisting them somewhere, and decrypts passwords into memory when server uses them, so that no more clear text secrets live in Tableau Server. The new service should support both our old monolith JRuby app and our new service-oriented server architecture.

How did I build a custom dynamic ZooKeeper management and migration framework and an asynchronous job framework for Tableau Server 11

 

First of all, Tableau Server 11 is not out yet. Tableau Server 10 is just released yesterday.

Product Strategy Overview

Tableau Server is an enterpise data analytics and visualization platform, and our customers are business people. In order to make the project usage experience as smooth as possible, we believe we should enable business people to manage Tableau Server on their own without any help from enterprise IT.

Therefore, one of the most critical goals for developing Tableau Server is that Tableau will provide GUIs and built-in workflows to automate as many managing processes as possible.

Project and Feature Requirements

Tableau core server team is building a new Java-based Tableau Server architecutre for Tableau Server 11 (target release, might change in the future) to replace the old jruby one.

Notably, old jruby Tabadmin persisting cluster-wide configuration and topology information in a local file (workgroup.yml). The role of ZooKeeper (referenced as ZK) is only used a distributed lock and a temp data store – it keeps some vilatile data that can be lost or wiped out and no one cares about it. This characteristic also makes managing and migrating ZooKeeper very very easy – you only need to clean up all data in existing ZooKeeper nodes, deploy the ZooKeeper bits to new nodes, and bring the new cluster up.

Unlike the old world, new Tabadmin will persist all essential configuration and topology data in ZK as a single source of truth, which makes managing, expanding, and migrating ZK much harder than before.

The Problem

The problem we are facing is how to maintain ZK’s data and expand/shrink ZK cluster, as well as making this process super reliable and easy for users.

Let’s elaborate more about the problem.

Say we start with a single Tableau Server node A, which has a single node ZK cluster runs on it. Then we want to expand the Tableau Server cluster from 1 node (A) to 3 nodes (A, B, C), as well as the ZK cluster runs on top of it. How to achieve that?

(A) -> (A, B, C)

Non-Goal

For customers who already have a ZK cluster in house and want to reuse that cluster, Tableau Server provides configurations that enable them to do so. That is not part of the project, and will not be discussed here. This project is focusing on smoothing the experience of managing Tableau Server’s built-in ZooKeeper cluster.

Naive Approach

The naive approach is to shut down (A), deploy ZK bits on B and C, change their configuration files, and bring up (A, B, C). Well, this can lead to disasters. The reason being that ZK runs on a quorum mode, means that if more than half nodes in a ZK cluster form a quorum, their decision will be the final decision that all nodes should follow. In this case, if B and C come up first and form a quorum, their decision will be “there’s no data in our cluster”. When A joins them, B and C will force A to delete its data, causing data loss.

ZooKeeper’s built-in Approach

Actually all developers using ZooKeeper faces the same problem. Think about running a ZooKeeper in your data center. If a ZooKeeper node fails, the common practice is that admin will try to bring it back manually/through scripts. If the machine dies, admins have to bring up a new machine, change its IP to match the failed one, and join it to the cluster. And you must know that machine failure is more than normal.

So ZooKeeper 3.5 starts to build a native rolling based approach to solve this common problem. The principle is to bring up a new node each time, join it to the existing cluster, wait them to sync up and become stable, bring up another new node.

(A) -> (A, B) -> (A, B, C)

The problem is that ZK is not originally developed to support this use case. It involves lots of refactoring, produces lots of bugs, and is not stable for production.

Take a look at ZK’s release page. 3.5.0 was released around 3 years ago. Apache then release 3.5.1 and 3.5.2, both are alpha version. They themselves even don’t recommend using it for production.

 

How did I build a Facebook Messenger chatbot for Tableau’s ‘VizOfTheDay’ feed

What is Viz Of The Day

It’s a Tableau Public product used mainly for promoting Tableau as a brand.

https://public.tableau.com/s/gallery

What is Facebook Messenger chatbot

https://developers.facebook.com/blog/post/2016/04/12/bots-for-messenger/

What can my bot do?

• Users can query VizOfTheDay on any date through either a hardcoded format, or natural language (which will be processed by wit.ai). If the bot doesn’t find that or cannot recognize the message, it will return a random one for users.
• Users can subscribe to VizOfTheDay, so that when a new viz is published on Tableau Public, users will be notified on Messenger App by a push notification. Of course, they can also unsubscribe it.

How does the system look like

• Backend running on a single AWS EC2
• ELB as gateway, to bind a domain name for https
• Using DynamoDB for storage
• Calling Facebook Messenger API to create UI
• Taking advantages of wit.ai for some natural language processing

Modules and their Responsibilities:

• FeedProcessor
• periodically retrieves VizOfTheDay from a Tableau Public endpoint
• serves queries of viz on a specified date (can be today or any date before)
• Controller
• receives, processes, and interprets requests from Facebook Messenger
• distributes interpretted requests to corresponding services
• Response Service
• Takes responsibilities of replying requests
• Supports different kind of templates, from pure text, pure thumbnails, to a combination of text thumbnails, and multiple tabs
• User Service
• Stores and serves queries on users’ meta data (id, first and last name, etc) and subscription options (subscribed or not)
• Data is cached in memory for faster access to properties like first name and subscription option
• Data is persisted in DynamoDB
• Profile Querying Service
• Takes a user id and query its meta data against Facebook Graph API
• Push Notifier Service
• Runs daily
• Once detects a new viz from FeedProcessor, push it to all subscribed users by getting users from User Service

Facebook Messenger API
|
|
AWS ELB
|
|
AWS EC2
|
|       for interpretation
Controller ----------- wit.ai
|                                         |
|                                         |
------------------------|
enqueue id | |
| |
| |
DynamoDB -- User Service ------ Response Service
| | |
| | |
Profile Querying Serivce | |
| |
Push Notifier |
| |
| |
| |
Feed Processor----------

Hexo – How to install Hexo on Mac with github pages

 

1. Install node.js and git

• Install node.js from nodojs.org or by npm (package manager)
• Install git from git-scm.com

2. Setup Github and ssh keys

Check your local ssh keys

$ cd ~/. ssh

If it reports: No such file or directory, you are using git for the first time.

Generate new ssh key

$ ssh-keygen -t rsa -C "email@host.com"

it will show

Generating public/private rsa key pair....
Enter passphrase... # enter your passphrase
Enger same passphrase again: #enter it again

Add the ssh key to github

Add the key in .ssh/id_rsa.pub file to your github account

Test

$ ssh -T git@github.com

If succeed, it should show

The authenticity of host 'github.com' can't be established.
RSA key fingerprint is xxxx
Are you sure you want to continue connecting (yes/no)? # yes

Finally, it shows

You've successfully authenticated, but GitHub does not provide shell access.

Setup user account

$ git config --global user.name "username"
$ git config --global user.email "email@host.com"

<!– more –>

3. Install Hexo on mac

Install Hexo on your mac

npm install hexo-cli --save
npm install hexo --save

Init Hexo to a dir

Create a dir to hold Hexo for blogging, and Hexo will generate all static files automatically to that dir

$ cd your_dir
$ hexo init

Start Hexo

$ hexo g
$ hexo s

Deploy Hexo to github

Before running this, you need to look at the later section of how to configure Hexo

$ hexo deploy

It may shows the following error:

ERROR Deployer not found: github

That's because you don't the deployer plug. Install one by running

$ npm install hexo-deployer-git --save

One problem I ran into is, once when I ran the above command, console reported some weird errors complaining about headers. After a while, I figured out I typed Chinese's '-' instead of English's '-'…

Feed

$ npm install hexo-generator-feed --save

And add the following to Hexo's _config.yml file

#Feed Atom
feed:
type: atom
path: atom.xml
limit: 20
hub:

<h2></h2>
<!–more–>
<h2>SEO</h2>

$ npm install hexo-generator-sitemap --save
$ npm install hexo-generator-baidu-sitemap --save

And add the following to Hexo's _config.yml file

#sitemap
sitemap:
path: sitemap.xml
baidusitemap:
path: baidusitemap.xml

Hexo theme – next

git clone https://github.com/iissnan/hexo-theme-next themes/next

Shortcut

npm install hexo --save
npm install hexo-cli --save
npm install hexo-deployer-git --save
npm install hexo-generator-feed --save
npm install hexo-generator-sitemap --save
npm install hexo-generator-baidu-sitemap --save
npm install hexo-generator-search --save
git clone https://github.com/iissnan/hexo-theme-next themes/next

My Hexo template

• Keep in mind that Hexo and Next will both evolve. So next time when setting up config, be cautious and not to just copy and past the following config files.

# Hexo Configuration
## Docs: https://hexo.io/docs/configuration.html
## Source: https://github.com/hexojs/hexo/
# Site
title: Bowen's blog
subtitle: keep learning - learning notes and blogs
description: keep learning - learning notes and blogs
author: Bowen
email:
language: en
timezone:
# Avatar
avatar: /avatar/avatar.jpg
# URL
## If your site is put in a subdirectory, set url as 'http://yoursite.com/child' and root as '/child/'
url: https://phoenixjiangnan.github.io
root: /
permalink: :year/:month/:day/:title/
permalink_defaults:
# Directory
source_dir: source
public_dir: public
tag_dir: tags
archive_dir: archives
category_dir: categories
code_dir: downloads/code
i18n_dir: :lang
skip_render:
# Writing
new_post_name: :title.md # File name of new posts
default_layout: post
titlecase: false # Transform title into titlecase
external_link: true # Open external links in new tab
filename_case: 0
render_drafts: false
post_asset_folder: false
relative_link: false
future: true
highlight:
enable: true
line_number: false
auto_detect: false
tab_replace:

# Category & Tag
default_category: uncategorized
category_map:
tag_map:
# Archives
## 2: Enable pagination
## 1: Disable pagination
## 0: Fully Disable
archive: 2
category: 2
tag: 2
# Date / Time format
## Hexo uses Moment.js to parse and display date
## You can customize the date format as defined in
## http://momentjs.com/docs/#/displaying/format/
date_format: YYYY-MM-DD
time_format: HH:mm:ss
# Disqus
disqus_shortname: bowensgithubblog
# Markdown
## https://github.com/chjj/marked
markdown:
gfm: true
pedantic: false
sanitize: false
tables: true
breaks: true
smartLists: true
smartypants: true
#sitemap
sitemap:
path: sitemap.xml
baidusitemap:
path: baidusitemap.xml
#Feed Atom
feed:
type: atom
path: atom.xml
limit: 20
hub:
# Pagination
## Set per_page to 0 to disable pagination
per_page: 20
pagination_dir: page
# Extensions
## Plugins: https://hexo.io/plugins/
## Themes: https://hexo.io/themes/
theme: next
# Plugins
# exclude_generator:
# Plugins:
# - hexo-generator-feed
# - hexo-generator-sitemap
# Deployment
## Docs: https://hexo.io/docs/deployment.html
deploy:
type: git
repository: https://github.com/phoenixjiangnan/phoenixjiangnan.github.io.git
branch: master

My Next template

The config file is /Hexo/themes/next/_config.yml

# ---------------------------------------------------------------
# Site Information Settings
# ---------------------------------------------------------------

# Put your favicon.ico into `hexo-site/source/` directory.
favicon: /favicon.ico

# Set default keywords (Use a comma to separate)
keywords: "Hexo, welcome to Bowen's blog"

# Set rss to false to disable feed link.
# Leave rss as empty to use site's feed link.
# Set rss to specific value if you have burned your feed already.
rss:

# Specify the date when the site was setup
#since: 2015

# Canonical, set a canonical link tag in your hexo, you could use it for your SEO of blog.
# See: https://support.google.com/webmasters/answer/139066
# Tips: Before you open this tag, remeber set up your URL in hexo _config.yml ( ex. url: http://yourdomain.com )
canonical: true
# ---------------------------------------------------------------
# Menu Settings
# ---------------------------------------------------------------

# When running the site in a subdirectory (e.g. domain.tld/blog), remove the leading slash (/archives -> archives)
menu:
home: /
about: https://phoenixjiangnan.github.io/about
archives: https://phoenixjiangnan.github.io/archives
categories: https://phoenixjiangnan.github.io/categories
tags: https://phoenixjiangnan.github.io/tags
#commonweal: /404.html
# Enable/Disable menu icons.
# Icon Mapping:
# Map a menu item to a specific FontAwesome icon name.
# Key is the name of menu item and value is the name of FontAwsome icon. Key is case-senstive.
# When an question mask icon presenting up means that the item has no mapping icon.
menu_icons:
enable: true
#KeyMapsToMenuItemKey: NameOfTheIconFromFontAwesome
home: home
about: user
categories: th
tags: tags
archives: archive
commonweal: heartbeat
# ---------------------------------------------------------------
# Scheme Settings
# ---------------------------------------------------------------

# Schemes
scheme: Muse
#scheme: Mist
#scheme: Pisces
# ---------------------------------------------------------------
# Font Settings
# - Find fonts on Google Fonts (https://www.google.com/fonts)
# - All fonts set here will have the following styles:
# light, light italic, normal, normal intalic, bold, bold italic
# - Be aware that setting too much fonts will cause site running slowly
# - Introduce in 5.0.1
# ---------------------------------------------------------------
font:
enable: true

# Uri of fonts host. E.g. //fonts.googleapis.com (Default)
host:

# Global font settings used on <body> element.
global:
# external: true will load this font family from host.
external: true
family: Lato

# Font settings for Headlines (h1, h2, h3, h4, h5, h6)
# Fallback to `global` font settings.
headings:
external: true
family:

# Font settings for posts
# Fallback to `global` font settings.
posts:
external: true
family:

# Font settings for Logo
# Fallback to `global` font settings.
# The `size` option use `px` as unit
logo:
external: true
family:
size:

# Font settings for <code> and code blocks.
codes:
external: true
family:
# ---------------------------------------------------------------
# Sidebar Settings
# ---------------------------------------------------------------
# Social Links
# Key is the link label showing to end users.
# Value is the target link (E.g. GitHub: https://github.com/iissnan)
#social:
#LinkLabel: Link
# Social Links Icons
# Icon Mapping:
# Map a menu item to a specific FontAwesome icon name.
# Key is the name of the item and value is the name of FontAwsome icon. Key is case-senstive.
# When an globe mask icon presenting up means that the item has no mapping icon.
social_icons:
enable: true
# Icon Mappings.
# KeyMapsToSocalItemKey: NameOfTheIconFromFontAwesome
GitHub: github
Twitter: twitter
Weibo: weibo
# Sidebar Avatar
# in theme directory(source/images): /images/avatar.jpg
# in site directory(source/uploads): /uploads/avatar.jpg
#avatar:
# Table Of Contents in the Sidebar
toc:
enable: true

# Automatically add list number to toc.
number: true
# Creative Commons 4.0 International License.
# http://creativecommons.org/
# Available: by | by-nc | by-nc-nd | by-nc-sa | by-nd | by-sa | zero
#creative_commons: by-nc-sa
#creative_commons:
sidebar:
# Sidebar Position, available value: left | right
position: left
#position: right

# Sidebar Display, available value:
# - post expand on posts automatically. Default.
# - always expand for all pages automatically
# - hide expand only when click on the sidebar toggle icon.
# - remove Totally remove sidebar including sidebar toggler.
display: post
#display: always
#display: hide
#display: remove
# Blogrolls
#links_title: Links
#links_layout: block
#links_layout: inline
#links:
#Title: http://example.com/
# ---------------------------------------------------------------
# Misc Theme Settings
# ---------------------------------------------------------------

# Custom Logo.
# !!Only available for Default Scheme currently.
# Options:
# enabled: [true/false] - Replace with specific image
# image: url-of-image - Images's url
custom_logo:
enabled: false
image:
# Code Highlight theme
# Available value:
# normal | night | night eighties | night blue | night bright
# https://github.com/chriskempson/tomorrow-theme
highlight_theme: night eighties
# Automatically scroll page to section which is under <!-- more --> mark.
scroll_to_more: true
# Automatically Excerpt. Not recommand.
# Please use <!-- more --> in the post to control excerpt accurately.
auto_excerpt:
enable: false
length: 150
# Wechat Subscriber
#wechat_subscriber:
#enabled: true
#qcode: /path/to/your/wechatqcode ex. /uploads/wechat-qcode.jpg
#description: ex. subscribe to my blog by scanning my public wechat account
# ---------------------------------------------------------------
# Third Party Services Settings
# ---------------------------------------------------------------

# MathJax Support
mathjax:
enable: false
cdn: //cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML
# Swiftype Search API Key
#swiftype_key:

# Baidu Analytics ID
#baidu_analytics:

# Duoshuo ShortName
#duoshuo_shortname:

# Disqus
#disqus_shortname:

# Baidu Share
# Available value:
# button | slide
#baidushare:
## type: button

# Share
#jiathis:
#add_this_id:

# Share
#duoshuo_share: true

# Google Webmaster tools verification setting
# See: https://www.google.com/webmasters/
#google_site_verification:
# Google Analytics
#google_analytics:

# CNZZ count
#cnzz_siteid:
# Make duoshuo show UA
# user_id must NOT be null when admin_enable is true!
# you can visit http://dev.duoshuo.com get duoshuo user id.
duoshuo_info:
ua_enable: true
admin_enable: false
user_id: 0
#admin_nickname: Author
# Facebook SDK Support.
# https://github.com/iissnan/hexo-theme-next/pull/410
facebook_sdk:
enable: false
app_id: #<app_id>
fb_admin: #<user_id>
like_button: #true
webmaster: #true

# Facebook comments plugin
# This plugin depends on Facebook SDK.
# If facebook_sdk.enable is false, Facebook comments plugin is unavailable.
facebook_comments_plugin:
enable: false
num_of_posts: 10 # min posts num is 1
width: 100% # default width is 550px
scheme: light # default scheme is light (light or dark)
# Show number of visitors to each article.
# You can visit https://leancloud.cn get AppID and AppKey.
leancloud_visitors:
enable: false
app_id: #<app_id>
app_key: #<app_key>

# Show PV/UV of the website/page with busuanzi.
# Get more information on http://ibruce.info/2015/04/04/busuanzi/
busuanzi_count:
# count values only if the other configs are false
enable: false
# custom uv span for the whole site
site_uv: true
site_uv_header: <i class="fa fa-user"></i>
site_uv_footer:
# custom pv span for the whole site
site_pv: true
site_pv_header: <i class="fa fa-eye"></i>
site_pv_footer:
# custom pv span for one page only
page_pv: true
page_pv_header: <i class="fa fa-file-o"></i>
page_pv_footer:

# Tencent analytics ID
# tencent_analytics:

# Enable baidu push so that the blog will push the url to baidu automatically which is very helpful for SEO
baidu_push: false

#! ---------------------------------------------------------------
#! DO NOT EDIT THE FOLLOWING SETTINGS
#! UNLESS YOU KNOW WHAT YOU ARE DOING
#! ---------------------------------------------------------------

# Motion
use_motion: true

# Fancybox
fancybox: true
# Script Vendors.
# Set a CDN address for the vendor you want to customize.
# For example
# jquery: https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js
# Be aware that you should use the same version as internal ones to avoid potential problems.
vendors:
# Internal path prefix. Please do not edit it.
_internal: vendors

# Internal version: 2.1.3
jquery:

# Internal version: 2.1.5
# http://fancyapps.com/fancybox/
fancybox:
fancybox_css:

# Internal version: 1.0.6
# https://github.com/ftlabs/fastclick
fastclick:

# Internal version: 1.9.7
# https://github.com/tuupola/jquery_lazyload
lazyload:

# Internal version: 1.2.1
# http://VelocityJS.org
velocity:

# Internal version: 1.2.1
# http://VelocityJS.org
velocity_ui:

# Internal version: 0.7.9
# https://faisalman.github.io/ua-parser-js/
ua_parser:

# Internal version: 4.4.0
# http://fontawesome.io/
fontawesome:
# Assets
css: css
js: js
images: images

# Theme version
version: 5.0.1

My Post Template

The templates are in /Hexo/scaffolds/*

The Post template post.md is

---
title: {{ title }}
date: {{ date }}
categories: []
tags: []
---

---

 

Reference

• http://www.jianshu.com/p/05289a4bc8b2