Projects

Length Controllable Visual Storytelling
Distilled nouns and verbs from a sequence of images and utilized knowledge graph to find the important relations between nouns. Dynamically performed recurrent Transformer to generated stories with diverse length. The human evaluation showed that our model can generate longer stories, even when the input images are incohert.
[Work in Progress]

Knowledge-Enriched Visual Storytelling (KG-Story)
KG-Story, a three-stage framework that allows the story generation model to take advantage of external Knowledge Graphs to produce interesting stories. KG-Story distills a set of representative words from the input prompts, enriches the word set by using ex-ternal knowledge graphs, and finally generates stories based on the enriched word set.
[Paper Link]

Emoji Prediction
Hashtags and Application Sources like Android, etc. are two features which we found to be important yet underused in emoji prediction and Twitter sentiment analysis on the whole. We showcase the importance of using Twitter features to helpthe model understand the sentiment involved and hence to predict the most suitable emoji for the text. To further understand emoji behavioral patterns, we propose a more balanced dataset by crawling additional Twitter data, including timestamp, hashtags, and application source acting as additional attributes to the tweet.
[Class Project]

Predicting Crop Price Trends using a Deep Learning Approach
Farmer suicides have become an urgent social problem that governments around the world are trying hard to solve. Most farmers are driven to suicide due to an inability to sell their produce at desired profit levels, which is caused by the widespread uncertainty/fluctuation in produce prices resulting from varying market conditions. To help the farmers with the issue of produce price uncertainty, this paper proposes a deep learning algorithm for prediction of future produce price trends (Increase, Decrease, Stable) based on past pricing and volume pattern.
[Paper Link]

Automatic Caption Generation for Twitter Disaster Scene
Twitter is a mainstream social media platform for users to share information. Inparticular, during the disaster, there are large volume of tweets posted on Twitter withvarious kinds of contents. Fortunately, Twitter provides api that allows users to crawlthe images and captions from their database. Based on the given data, we proposed an image captioning model to generate textual descriptions for disaster-related images.
[Class Project]