This is a quick question. I am reading a mass amount of JSON from a variety of text files. The JSONs are tweets, in this format:
{"filter_level":"medium","retweeted_status":{"contributors":null,"text":"Daily Mail: Vincent Kompany says Manchester United are favourites http://t.co/NB2p1cqlRs #mufc","geo":null,"retweeted":false,"in_reply_to_screen_name":null,"possibly_sensitive":false,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[{"expanded_url":"http://dailym.ai/15y9p6X","indices":[66,88],"display_url":"dailym.ai/15y9p6X","url":"http://t.co/NB2p1cqlRs"}],"hashtags":[{"text":"mufc","indices":[89,94]}],"user_mentions":[]},"in_reply_to_status_id_str":null,"id":364326884395323393,"source":"<a href=\"http://twitterfeed.com\" rel=\"nofollow\">twitterfeed<\/a>","in_reply_to_user_id_str":null,"favorited":false,"in_reply_to_status_id":null,"retweet_count":9,"created_at":"Mon Aug 15 10:07:52 +0000 2013","in_reply_to_user_id":null,"favorite_count":0,"id_str":"364326884395323393","place":null,"user":{"location":"Old Trafford, Manchester, M16 ","default_profile":false,"profile_background_tile":true,"statuses_count":20839,"lang":"en","profile_link_color":"BD0015","profile_banner_url":"https://pbs.twimg.com/profile_banners/16557902/1347990848","id":16557902,"following":null,"protected":false,"favourites_count":0,"profile_text_color":"333333","description":"Tweeting the latest Manchester United news from the BBC and a number of other sources. Unofficial. Created by @gavreilly.","verified":false,"contributors_enabled":false,"profile_sidebar_border_color":"FFFFFF","name":"BBC Manchester Utd","profile_background_color":"BD0015","created_at":"Thu Oct 02 11:24:52 +0000 2008","default_profile_image":false,"followers_count":56086,"profile_image_url_https":"https://si0.twimg.com/profile_images/61130289/manutd_normal.png","geo_enabled":true,"profile_background_image_url":"http://a0.twimg.com/profile_background_images/662809119/lj2ujpnj7iwwvvxxpoor.jpeg","profile_background_image_url_https":"https://si0.twimg.com/profile_background_images/662809119/lj2ujpnj7iwwvvxxpoor.jpeg","follow_request_sent":null,"url":"http://news.bbc.co.uk/sport2/hi/football/teams/m/man_utd/default.stm","utc_offset":3600,"time_zone":"London","notifications":null,"profile_use_background_image":true,"friends_count":54572,"profile_sidebar_fill_color":"FFC8C7","screen_name":"bbcmanutd","id_str":"16557902","profile_image_url":"http://a0.twimg.com/profile_images/61130289/manutd_normal.png","listed_count":789,"is_translator":false},"coordinates":null},"contributors":null,"text":"RT @bbcmanutd: Daily Mail: Vincent Kompany says Manchester United are favourites http://t.co/NB2p1cqlRs #mufc","geo":null,"retweeted":false,"in_reply_to_screen_name":null,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[],"hashtags":[{"text":"mufc","indices":[104,109]}],"user_mentions":[{"id":16557902,"name":"BBC Manchester Utd","indices":[3,13],"screen_name":"bbcmanutd","id_str":"16557902"}]},"in_reply_to_status_id_str":null,"id":364329399950540800,"source":"<a href=\"http://blackberry.com/twitter\" rel=\"nofollow\">Twitter for BlackBerry®<\/a>","in_reply_to_user_id_str":null,"favorited":false,"in_reply_to_status_id":null,"retweet_count":0,"created_at":"Mon Aug 15 10:17:52 +0000 2013","in_reply_to_user_id":null,"favorite_count":0,"id_str":"364329399950540800","place":null,"user":{"location":"","default_profile":false,"profile_background_tile":false,"statuses_count":3520,"lang":"en","profile_link_color":"2FC2EF","profile_banner_url":"https://pbs.twimg.com/profile_banners/561283874/1375643576","id":561283874,"following":null,"protected":false,"favourites_count":234,"profile_text_color":"666666","description":"A curious case. An enigma. ..","verified":false,"contributors_enabled":false,"profile_sidebar_border_color":"181A1E","name":"vi̲̅ƞci̲̅ thE EnIgma","profile_background_color":"1A1B1F","created_at":"Mon Apr 23 16:41:58 +0000 2012","default_profile_image":false,"followers_count":195,"profile_image_url_https":"https://si0.twimg.com/profile_images/378800000241332248/569425827c3cc3ad8ff44958bad5307a_normal.jpeg","geo_enabled":false,"profile_background_image_url":"http://a0.twimg.com/images/themes/theme9/bg.gif","profile_background_image_url_https":"https://si0.twimg.com/images/themes/theme9/bg.gif","follow_request_sent":null,"url":null,"utc_offset":7200,"time_zone":"Amsterdam","notifications":null,"profile_use_background_image":true,"friends_count":339,"profile_sidebar_fill_color":"252429","screen_name":"Dinnydavinci","id_str":"561283874","profile_image_url":"http://a0.twimg.com/profile_images/378800000241332248/569425827c3cc3ad8ff44958bad5307a_normal.jpeg","listed_count":0,"is_translator":false},"coordinates":null}
I am reading them with the following code:
public void performAnalysis() {
File tweets;
String[] categories;
tweets = new File("//home//andrew//Python//Tweets//");
categories = tweets.list();
for (int i = 0; i < categories.length; ++i) {
String category = categories[i];
File file = new File(tweets, category);
try {
Scanner sc = new Scanner(file);
while (sc.hasNextLine()) {
String rawTweet = sc.nextLine().trim();
if (rawTweet.equals(""))
continue;
Tweet parsedTweet = new Gson().fromJson(rawTweet, Tweet.class);
System.out.println(parsedTweet.getText());
}
sc.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
I have another class called Tweet, with one instance variable: text. The Google Gson library allows me to strip the field "text" from the JSON which I am importing.
My problem is regarding the if statement in the while loop. It seems to be necessary. I didn't realise I had whitespace after my JSONs, but the program crashes at the end of each file it reads if I don't include this.
Does anybody have any suggestions on how to check the text file for white space or get rid of it without this if statement? Maybe I'm totally wrong, but it just looks horrible to me - especially since I'm saving the line, trimming it, THEN checking if it's empty. I was going to check first, but I don't know how without using sc.nextLine() twice (thus skipping a line).
