Every day, Duga reports the number of unanswered questions and percentage of answered questions (after she calls the API endpoint /info) in The 2nd Monitor (so we can keep the Zombies at bay).

I created a script to read the messages by her in the past few months and plot the percentage of answered questions on a line chart. A demonstration of it can be seen in this playground example.

What would you change? Obviously there are other techniques for fetching and parsing the data... but is this approach fine?

<?php
$dd = new DomDocument();$internalErrors = libxml_use_internal_errors(true);
$html = file_get_contents('https://chat.stackexchange.com/search?q=%22RELOAD!+There+are%22&user=125580&room=8595&pagesize=150');$data = [];
$labels = []; if ($html) {
$dd->loadHtml($html);
$xml = simplexml_import_dom($dd);

$messages =$xml->xpath('//div[@class="content"]');
$timestamps =$xml->xpath('//div[@class="timestamp"]');

if (count($messages) == count($timestamps)) {
foreach(array_reverse($timestamps, true) as$index => $timestamp) { preg_match('#(\d{2}.\d{4})#',(string)$messages[$index],$matches);
if (count($matches) > 1) {$labels[] = (string)$timestamp;$data[] = $matches[1]; } } } ?> <html> <head> <script src="//www.chartjs.org/dist/2.7.2/Chart.bundle.js"></script> <script src="//www.chartjs.org/samples/latest/utils.js"></script> </head> <body> <div style="width:75%;"> <canvas id="canvas"></canvas> </div> <script type="text/javascript"> Chart.defaults.global.legend.display = false; const config = { type: 'line', data: { labels: <?=json_encode($labels);?>,
datasets: [{
backgroundColor: window.chartColors.red,
borderColor: window.chartColors.red,
data: <?=json_encode($data);?>, fill: false }] }, options: { title: { display: true, text: 'Answered percentage in recent months' }, scales: { xAxes: [{ display: true, scaleLabel: { display: true, labelString: 'Date' } }], yAxes: [{ display: true, scaleLabel: { display: true, labelString: 'Percentage' } }] } } }; document.addEventListener('DOMContentLoaded', _ => { var ctx = document.getElementById('canvas').getContext('2d'); window.myLine = new Chart(ctx, config); }); </script> </body> </html> <?php } else { echo 'Unable to fetch messages'; } ?>  Sample output: Chart.defaults.global.legend.display = false; const config = { type: 'line', data: { labels: ["Mar 11 12:00 AM", "Mar 12 12:00 AM", "Mar 13 12:00 AM", "Mar 14 12:00 AM", "Mar 15 12:00 AM", "Mar 16 12:00 AM", "Mar 17 12:00 AM", "Mar 18 12:00 AM", "Mar 19 12:00 AM", "Mar 20 12:00 AM", "Mar 21 12:00 AM", "Mar 22 12:00 AM", "Mar 23 12:00 AM", "Mar 24 12:00 AM", "Mar 25 12:00 AM", "Mar 26 12:00 AM", "Mar 27 12:00 AM", "Mar 28 12:00 AM", "Mar 29 12:00 AM", "Mar 30 12:00 AM", "Mar 31 12:00 AM", "Apr 1 12:00 AM", "Apr 2 12:00 AM", "Apr 3 12:00 AM", "Apr 4 12:00 AM", "Apr 5 12:00 AM", "Apr 6 12:00 AM", "Apr 7 12:00 AM", "Apr 8 12:00 AM", "Apr 9 12:00 AM", "Apr 10 12:00 AM", "Apr 11 12:00 AM", "Apr 12 12:00 AM", "Apr 13 12:00 AM", "Apr 14 12:00 AM", "Apr 15 12:00 AM", "Apr 16 12:00 AM", "Apr 17 12:00 AM", "Apr 18 12:00 AM", "Apr 19 12:00 AM", "Apr 20 12:00 AM", "Apr 22 12:00 AM", "Apr 23 12:00 AM", "Apr 24 12:00 AM", "Apr 25 12:00 AM", "Apr 26 12:00 AM", "Apr 27 12:00 AM", "Apr 28 12:00 AM", "Apr 29 12:00 AM", "Apr 30 12:00 AM", "May 1 12:00 AM", "May 2 12:00 AM", "May 3 12:00 AM", "May 4 12:00 AM", "May 5 12:00 AM", "May 6 12:00 AM", "May 7 12:00 AM", "May 8 12:00 AM", "May 9 12:00 AM", "May 10 12:00 AM", "May 11 12:00 AM", "May 12 12:00 AM", "May 13 12:00 AM", "May 14 12:00 AM", "May 15 12:00 AM", "May 16 12:00 AM", "May 17 12:00 AM", "May 18 12:00 AM", "May 19 12:00 AM", "May 20 12:00 AM", "May 21 12:00 AM", "May 22 12:00 AM", "May 23 12:00 AM", "May 24 12:00 AM", "May 25 12:00 AM", "May 26 12:00 AM", "May 27 12:00 AM", "May 28 12:00 AM", "May 29 12:00 AM", "May 30 12:00 AM", "May 31 12:00 AM", "Jun 1 12:00 AM", "Jun 2 12:00 AM", "Jun 3 12:00 AM", "Jun 4 12:00 AM", "Jun 5 12:00 AM", "Jun 6 12:00 AM", "Jun 7 12:00 AM", "Jun 8 12:00 AM", "Jun 9 12:00 AM", "Jun 10 12:00 AM", "Jun 11 12:00 AM", "Jun 12 12:00 AM", "Jun 13 12:00 AM", "Jun 14 12:00 AM", "Fri 12:00 AM", "Sat 12:00 AM", "Sun 12:00 AM", "Mon 12:00 AM", "yst 12:00 AM", "12:00 AM"], datasets: [{ label: 'Answered Percentage', backgroundColor: window.chartColors.red, borderColor: window.chartColors.red, data: ["90.0229", "90.0027", "89.9802", "89.9748", "89.9868", "89.9923", "89.9685", "89.9861", "90.0081", "89.9973", "89.9849", "90.0006", "89.9790", "89.9817", "89.9911", "89.9979", "90.0127", "90.0040", "90.0019", "90.0069", "89.9896", "89.9731", "89.9829", "89.9804", "89.9781", "89.9643", "89.9741", "89.9783", "89.9808", "89.9760", "89.9895", "90.0025", "90.0084", "89.9738", "89.9727", "90.0107", "90.0017", "90.0162", "90.0069", "90.0036", "90.0080", "90.0120", "90.0086", "90.0099", "90.0004", "90.0032", "89.9888", "89.9899", "89.9688", "89.9671", "89.9620", "89.9778", "89.9668", "89.9790", "89.9968", "90.0233", "90.0313", "90.0419", "90.0261", "90.0170", "90.0119", "90.0255", "90.0359", "90.0234", "90.0302", "90.0006", "90.0009", "89.9923", "89.9883", "90.0026", "90.0047", "89.9938", "90.0006", "89.9940", "89.9750", "89.9927", "90.0197", "90.0032", "90.0248", "90.0182", "90.0248", "90.0349", "90.0272", "90.0409", "90.0289", "90.0345", "90.0425", "90.0421", "90.0376", "90.0264", "90.0095", "89.9877", "89.9987", "89.9850", "89.9815", "89.9815", "89.9681", "89.9765", "89.9815", "89.9830", "89.9776"], fill: false }] }, options: { title: { display: true, text: 'Answered percentage in recent months' }, scales: { xAxes: [{ display: true, scaleLabel: { display: true, labelString: 'Date' } }], yAxes: [{ display: true, scaleLabel: { display: true, labelString: 'Percentage' } }] } } }; document.addEventListener('DOMContentLoaded', _ => { var ctx = document.getElementById('canvas').getContext('2d'); window.myLine = new Chart(ctx, config); }); <script src="//www.chartjs.org/dist/2.7.2/Chart.bundle.js"></script> <script src="//www.chartjs.org/samples/latest/utils.js"></script> <div style="width:75%;"> <div class="chartjs-size-monitor" style="position: absolute; left: 0px; top: 0px; right: 0px; bottom: 0px; overflow: hidden; pointer-events: none; visibility: hidden; z-index: -1;"> <div class="chartjs-size-monitor-expand" style="position:absolute;left:0;top:0;right:0;bottom:0;overflow:hidden;pointer-events:none;visibility:hidden;z-index:-1;"> <div style="position:absolute;width:1000000px;height:1000000px;left:0;top:0"></div> </div> <div class="chartjs-size-monitor-shrink" style="position:absolute;left:0;top:0;right:0;bottom:0;overflow:hidden;pointer-events:none;visibility:hidden;z-index:-1;"> <div style="position:absolute;width:200%;height:200%;left:0; top:0"></div> </div> </div> <canvas id="canvas" width="481" height="240" class="chartjs-render-monitor" style="display: block; width: 481px; height: 240px;"></canvas> </div> 1 Answer When I ran your script and started digging through the dom, I noticed that Apr 21 12:00 AM was missing -- I wonder why that was. Anyhow, I wanted to encourage you to adjust the timestamp values because they are irregularly formatted. Then I started to play with iterated strtotime() calls to generate consistent Y-m-d stamps, but some of the strings needed to be prepared, so I wrote some conditionals and it just felt like hacky bloat. If you are happy with the time strings in your graph, I'll just leave that bit alone. As a matter of directness, I'll recommend: $dd->loadHTMLFile('https://chat.stackexchange.com/search?q=%22RELOAD!+There+are%22&user=125580&room=8595&pagesize=150');


Rather than the two step:

$html = file_get_contents('https://chat.stackexchange.com/search?q=%22RELOAD!+There+are%22&user=125580&room=8595&pagesize=150');$dd->loadHtml($html);  The regex block can be tightened up slightly. Write the preg_match() in a conditional so that you don't need to count(). A capture group is not necessary and a literal dot is more accurate than an [any character] dot. if (preg_match('#\d{2}\.\d{4}#', (string)$messages[$index],$match)) {
$labels[] = (string)$timestamp;
$data[] =$match[0];
}


These are very small adjustments, so I'd say your work is fine.

p.s. I did also toy with //div[@class="messages"] so that I was guaranteed to be processing pairs of data (by making subsequent xpath calls to isolate the content and timestamp child elements). However, I abandoned that process, because it was adding too much complexity to an originally simple task and the effort (mine & php's) didn't seem to be worth the validation gains.