What is Viz Of The Day
It’s a Tableau Public product used mainly for promoting Tableau as a brand.
What is Facebook Messenger chatbot
What can my bot do?
- Users can query VizOfTheDay on any date through either a hardcoded format, or natural language (which will be processed by wit.ai). If the bot doesn’t find that or cannot recognize the message, it will return a random one for users.
- Users can subscribe to VizOfTheDay, so that when a new viz is published on Tableau Public, users will be notified on Messenger App by a push notification. Of course, they can also unsubscribe it.
How does the system look like
- Backend running on a single AWS EC2
- ELB as gateway, to bind a domain name for https
- Using DynamoDB for storage
- Calling Facebook Messenger API to create UI
- Taking advantages of wit.ai for some natural language processing
Modules and their Responsibilities:
- periodically retrieves VizOfTheDay from a Tableau Public endpoint
- serves queries of viz on a specified date (can be today or any date before)
- receives, processes, and interprets requests from Facebook Messenger
- distributes interpretted requests to corresponding services
- Response Service
- Takes responsibilities of replying requests
- Supports different kind of templates, from pure text, pure thumbnails, to a combination of text thumbnails, and multiple tabs
- User Service
- Stores and serves queries on users’ meta data (id, first and last name, etc) and subscription options (subscribed or not)
- Data is cached in memory for faster access to properties like first name and subscription option
- Data is persisted in DynamoDB
- Profile Querying Service
- Takes a user id and query its meta data against Facebook Graph API
- Push Notifier Service
- Runs daily
- Once detects a new viz from FeedProcessor, push it to all subscribed users by getting users from User Service
Facebook Messenger API | | AWS ELB | | AWS EC2 | | for interpretation Controller ----------- wit.ai | | | | ------------------------| enqueue id | | | | | | DynamoDB -- User Service ------ Response Service | | | | | | Profile Querying Serivce | | | | Push Notifier | | | | | | | Feed Processor----------
Some Design Tradeoffs
Whether to cache users in memory
- Pros: faster queries, better performance
- Can the data fit in memory?
Yes, and it’s based on an estimation of user numbers * data size.
E.g. let’s assume the number of uses is 100,000, which is already much more than enough. For each single user, the metadata and its size are
– user id – long, 64 bit = 8 byte
– user names – string, assume it’s of 20 chars = 20 * 16 bits = 40 byte
– total size = 100,000 * (40 + 8) byte ~ 100,000 * 50 byte = 5,000,000 byte = 5 mb
Single machine or multiple
- A single machine is enough, based on estimation of memory size, network IO, disk, etc
- Developing a monolith code running on a single box is really agile