Twitter Sentiment Visualisations

A web app that uses data from Twitter combined with sentiment analysis and emotion detection to create a series of data visualisations to illustrate the happy and less happy locations, topics and times.

Modules

Several open sauce node modules have been developed and published on npm as part of this project

• sentiment-analysis - useses the AFINN-111 word list to calculate overall sentiment of a sentence
• fetch-tweets - fetches tweets from Twitter based on topic, location, timeframe or combination
• stream-tweets - streams live Tweets in real-time
• remove-words - removes all non-key words from a string sentence
• place-lookup - finds the latitude and longitude for any fuzzy place name using the Google Places API
• hp-haven-sentiment-analysis - A Node.js client library for HP Haven OnDemand Sentiment Analysis module
• haven-entity-extraction - Node.js client for HP Haven OnDemand Entity Extraction
• tweet-location - calculates the location from geo-tagged Tweets using the Twitter Geo API
• find-region-from-location - given a latitude and longitude calculates which region that point belongs in

Project Planning

• A set of user stories with acceptance criteria have been drawn up outlining what features the finished solution should have. They are also managed on the Trello Board

Running Locally

1. Prerequisites - You will need Node.js, MongoDB and git installed on your system. You will also need Gulp and Bower, which (once node is installed) you can install by running npm install gulp bower -g.
2. Get the files - Run: git clone https://github.com/Lissy93/twitter- sentiment-visualisation.git to clone the repo and then cd twitter-sentiment-visualisation to navigate into the directory.
3. Install dependencies - Run: npm install to populates the node _modules, and then bower install which will download the bower_components.
4. Set Config Run: npm run config to generate config\src\keys.coffee which you will need to populate with your API keys and save. Also check that your happy with the app config in config/src/app-config.coffee.
5. Build Project - Run npm run build to generate the compiled code from the source.
6. Start MongoDB - See instructions: Starting a MongoDB instance. You will need to run MongoDB in a separate terminal window.
7. Run the project - Run npm start then open your browser and navigate to http://localhost:8080

View detailed installation instructions

To run the tests: npm test or see the more test strategy

Automated Development Workflow

TSV uses the Gulp streaming build tool to automate the development workflow.

The key tasks you need to run are:

• gulp generate-config - before first-time running of the project, run this command to generate configuration files for your API keys
• gulp build - This will build the project fully, this includes cleaning the working directory and then all tasks that must happen for CoffeeScript, JavaScript, CSS, images, HTML and Browserify tasks.
• gulp nodemon - Runs the application on the default port (probably 8080)
• gulp test - This will run all unit and coverage tests, printing a summary of the results to the console and generating more detailed reports into the reports directory.
• gulp - this is the default task, it will check the project is configured correctly, build ALL the files, run the server, watch for changes, recompile relevant files and reload browsers on change, and keep all browsers in sync, when a test condition changes it will also re-run tests - a lot going on!

To read more about the project setup and gulp process, see build environment in the docs

Test Strategy

Twitter Sentiment Visualisation follows the TDD approach and is structured around it's unit tests.

To run tests: npm test

Testing Tools

• Framework - Mocha
• Assertion Library - Chai
• Coverage Testing - Istanbul
• Stubs, Spies and Mocking - Sinon.js
• Continuous Integration Testing - Travis CI
• Dependency Checking - David
• Automated Code Review's - Code Climate
• Headless Browser Testing <20> PhantomJS
• Testing HTTP services - SuperTest

More details on each of the tools and how they will be implemented along with the pass and fail criteria can be found on the test strategy page of the documentation.

Screen Shots

Click here to view the screen shots

The Stack

This project wouldn't have been possible at all without making use of many many open source packages, libraries, frameworks etc..

I would like to personally thank the hundreds of developers who have worked on open source packages like these.

There is an extensive stack of technologies that were used to develop the final application. The following list is a summary of the key utilities:

Documentation

Project Planning

User Stories

High Level UML

Methodology

Test Strategy

Development Documentation

Installation Instructions

Build Environment

Testing