2
\$\begingroup\$

So I created this custom application and ported it into a few websites. It uses the Twitch API and pulls in active streams and the viewer count. The problem I run into is when the list gets so big it seems to become too much to do though ajax. I think this because sometimes it will show a stream online when it has been offline for awhile. I'm looking to improve it so open to suggestions. I'll leave a link at the bottom to an simple active page I have set up.

This is the AJAX / JS to handle my channel info and set them up into the page. I store some flash parameters in here because I didn't find it possible to change flash vars via JS so I just reset it every time. When there's no active channels it falls back on showing youtube videos.

var channel = new Array();
var stats = new Array();
var title = new Array();
var viewers = new Array();
var called = true;
var active = false;
var query = window.location.search.split( '?' );
var current; // For viewers update
var ran;
var youtube = '<iframe width="560" height="315" frameborder="0" allowfullscreen></iframe>';

var tubes = ["zn7-fVtT16k", "7rE0-ek6MZA", "zj2Zf9tlg2Y", "mhTd4_Ids80", "AeNYDwbm9qw", "mgVwv0ZuPhM",
             "l3w2MTXBebg", "WA4tLCGcTG4", "TAaE7sJahiw", "xBzoBgfm55w", "AFA-rOls8YA", "YHRxv-40WMU",
             "ZIMoQHpvFQQ", "7ZsKqbt3gQ0", "1_hKLfTKU5Y", "UcTLJ692F70", "CeLrlmV9A-s", "FArZxLj6DLk",
             "gAYL5H46QnQ", "NisCkxU544c", "BBcYG_J3O2Q"];


function getList(){
var i=0;
$.getJSON(
    "streams.php",
    function(data) 
    {
        var first;

        while(data.streams[i]){
            channel[i] = data.streams[i];
            stats[i] = data.status[i];
            title[i] = data.title[i];
            viewers[i] = data.viewers[i];
            i++;
        }

        var online = $.inArray('online', stats);
        if(online != -1){
            first = online;
            current = online;
            online = true;
        }

        if(query[1] != null && called)
            specific();
        else if(online && called)
            build(first);
        else if(called)
            randomTube();

        addList();
        updateViewers();
    }
  );
}

getList();

function build(a){
    var data = 'http://www.twitch.tv/widgets/live_embed_player.swf?channel=';   // Object Data
    var src = 'hostname=www.twitch.tv&auto_play=true&start_volume=25&channel='; // Flashvars Param
    data += channel[a];
    src += channel[a];
    var changeVars = '<param name="flashvars" value="hostname=www.twitch.tv&auto_play=true&start_volume=25&channel='+src+'"/>';
    var params = '<param name="allowFullScreen" value="true" />' +
                 '<param name="allowScriptAccess" value="always" />' +
                 '<param name="allowNetworking" value="all" />' +
                 '<param name="movie" value="http://www.twitch.tv/widgets/live_embed_player.swf" />' +
                 changeVars;
    $("#twitchTV").html(params);
    $("#twitchTV").attr("data", data);
    if(stats[a] == 'online')
        $('#streamTitle').html("Streamer: " + title[a] + " - <span id=\"viewers\">" + viewers[a] + "</span> Viewers");
    else
        $('#streamTitle').text("Streamer: " + title[a] + " - Offline");

    current = a;
    called = false;
}

function addList(){
    var i = 0;
    var content = '';

    while(channel[i]){
        if(stats[i] == 'online'){
            content += '<li class="item"><div class="online"></div><a style="color:green" href="javascript: void(0)"'+
                       ' title="'+title[i]+' Stream : '+viewers[i]+' Viewers" onClick="changeStream($(this).text())">'+title[i]+'</a></li>';
        }
        i++;
    }
    i=0;
    while(channel[i]){
        if(stats[i] == 'offline'){
            content += '<li class="item"><div class="offline"></div><a class="keyWord" href="javascript: void(0)"'+
                       ' title="'+title[i]+' Stream" onClick="changeStream($(this).text())">'+title[i]+'</a></li>';
        }
        i++;
    }

    $('#list ul').html(content);
}

function updateViewers(){
    $('#viewers').text(viewers[current]);
}

function changeChat(){
    var chatSrc = $('#chatFrame').attr('src');
    var chat = chatSrc.split('channel=');
    $('#chatFrame').attr('src', chat[0]+'channel='+channel[current]);
    $('#mainChat').removeClass('activeChat');
    $('#streamChat').addClass('activeChat');
}

function changeStream(find){
    var found = $.inArray(find, title);
    if(found != -1){
            build(found);
            current = found;
    }

    if($('#streamChat').hasClass('activeChat'))
        changeChat();
}

function randomTube(){
    $('#player').html(youtube);
    $('#player iframe').attr("src", "http://www.youtube.com/embed/"+ tubes[Math.floor(Math.random()*tubes.length)]);
    $('#streamTitle').html('<a href="javascript: void(0)" title="Random Video" onClick="randomTube()" class="rano">Random Video</a>');
    called = false;
}

function specific(){
    var found = $.inArray(query[1], channel);
    if(found != -1){
        build(found);
        current = found;
        called = false;
    }
}

setInterval(getList, 1000);             // Update the streams list every second

The PHP file reads from a file which contains the channel name and channel Nick Name (which is how we want to show it in the webpage) and then I loop through to check and see which ones are active:

<?php
$streamer = array();
$status = array();
$title = array();
$viewers = array();
$i = 0;

$handle = file("streamers.txt", FILE_IGNORE_NEW_LINES);
foreach($handle as $chan){
        $temp = explode("-t-", $chan);                  // Seperate Streamer from Title
        $streamer[$i] = $temp[0];                               // Streamer is on left of -t-
        $title[$i] = $temp[1];                                  // Title is on the right of -t-
        $i++;
}

$i = 0;     
    foreach($streamer as $index)                                    // Loop through and see what streams are online
    {
        $json_file = @file_get_contents("http://api.justin.tv/api/stream/list.json?channel={$index}", 0, null, null);
        $json_array = json_decode($json_file, true);
        if ($json_array[0]['name'] == "live_user_{$index}")
        {
            $status[$i] = "online";
            $viewers[$i] = $json_array[0]['channel_count'];
        }
        else
        {
            $status[$i] = "offline";
            $viewers[$i] = 0;
        }
        $i++;
    }

    // Combine the 3 needed arrays to send through JSON
    $data['streams'] = $streamer;
    $data['status'] = $status;
    $data['title'] = $title;
    $data['viewers'] = $viewers;

    echo json_encode($data);

I run into a bottle neck once I have so many channels to check. The only solution I can see if to do check half then check another half, but I'm not sure. Here's some links to an active example. The Index Page, The JS Page and The what PHP Returns. Let me know if you have any questions!

\$\endgroup\$

2 Answers 2

2
\$\begingroup\$

Have you attempted to isolate the different functions? i.e. test the ajax against local data (just copy an output from the php page and hit that) instead of a php page that rebuilds the list every time? If you are still getting slow down issues than you know it is the javascript that is slow. If not you can be pretty sure that it is the querying of the api that is slow.

My guess is that your biggest bottleneck is this line (I have nothing to back this up, just a guess):

@file_get_contents("http://api.justin.tv/api/stream/list.json?channel={$index}", 0, null, null);

According to the docs the data in the list function is cached for 60 seconds. So no reason to poll more often than that. http://apiwiki.justin.tv/mediawiki/index.php/Stream/list. Polling more often is just wasting resources.

As an aside, as a general rule you should try to limit your use of global variables. Not sure if this would affect any of the performance problems so I won't go into specifics on that unless asked.

\$\endgroup\$
3
  • \$\begingroup\$ Global Javascript Variables or Global PHP Variables? And good call on the cache - I didn't realize that. I don't really see why that PHP line would cause a major issue since it is just getting the data. Probably lowing the call on the ajax request to 60s would help it out. \$\endgroup\$ Commented Nov 7, 2012 at 2:26
  • \$\begingroup\$ As a general rule you should avoid to use the suppress operator (@) to suppress errors. \$\endgroup\$
    – user555
    Commented Nov 7, 2012 at 2:39
  • \$\begingroup\$ I was referring to the javascript as I am more familiar with javascript and am not completely familiar with PHP conventions. If you are calling an API once per stream to get the info you are wiring up all the http calls for every stream. As a single call not a big deal but can get heavy with multiple calls. These are just guesses though, which I is why I recommended removing this and trying to load data with the js only. Isolate the parts to determine which is slowing you down. Otherwise you will only be guessing at what to fix. \$\endgroup\$
    – nickles80
    Commented Nov 7, 2012 at 2:57
2
+25
\$\begingroup\$

Updating the list through ajax every second takes a lot of resources, both client and server side. If you insist on doing the update so often I recommend you take a look at, comet or websockets. Read Is there some way to PUSH data from web server to browser?.

Also witch part of your applications seems to be the bottleneck? Is it the PHP script that is lagging behind or is it the AJAX side of the app that is not working as you would like?

\$\endgroup\$
5
  • \$\begingroup\$ That's what I was talking about is the resources it takes to update the list. I looked into web sockets but I tried to play around with it and maybe I don't understand how it works but I need to loop through a list to update it instead of just leaving open a port. It has to go through many links and many 'channels'. Comet Might be the answer but I need to look into it. \$\endgroup\$ Commented Nov 5, 2012 at 8:06
  • 1
    \$\begingroup\$ Am I understanding you correctly that it's the ajax page that is using to much resources? If so I'm not supprised since pulling the server every second takes a lot of processing power. \$\endgroup\$
    – user555
    Commented Nov 5, 2012 at 8:45
  • \$\begingroup\$ Yeah. When it's updating the list of active channels (via JS that it pulls from the ajax data) that's when I run into problems so it's safe to assume my ajax requests are fairly heavy. What sucks though is I would love to check out Web Sockets and dealing with Persistent Connections but apparently my provider doesn't allow it. So while it may be a valid assumption that Web Sockets would improve my application I have no way of testing :/ \$\endgroup\$ Commented Nov 5, 2012 at 19:08
  • \$\begingroup\$ Do you need to pull the server every second? Try pulling every minute or so instead. I doubt it's mission critical if you update the channel list less often. \$\endgroup\$
    – user555
    Commented Nov 5, 2012 at 21:33
  • \$\begingroup\$ It probably doesn't need to pull the list every second but once a channel is active it needs to pull that channel (where web sockets would come in handy) to update the number of viewers viewing the active stream every second. Though even pulling the list every minute I imagine would still be fairly resource intensive. I'll run it through and see what happens though, again maybe splitting up the active channel into its own ajax call to get just viewers might be a good idea. \$\endgroup\$ Commented Nov 6, 2012 at 2:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.