4
\$\begingroup\$

I've recently added a custom Alexa Flask Briefing skill that calls an API endpoint which scrapes top news headlines from the Planet Python RSS feed.

It works - whenever you ask Alexa about the news, it's gonna read top 5 (flash briefing API's limitation) topics from the Planet Python section. But, I'd like to see if there are things to improve in the code:

import logging

from bs4 import BeautifulSoup
from lxml import etree
from dateutil.parser import parse
from flask import Flask, jsonify
from nltk import sent_tokenize
import requests


PLANET_PYTHON_FEED_URL = "http://planetpython.org/rss20.xml"

logger = logging.getLogger()
logger.setLevel(logging.INFO)

app = Flask(__name__)


@app.route("/")
def main():
    with requests.Session() as session:
        session.headers = {
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
        }

        response = session.get(PLANET_PYTHON_FEED_URL)
        data = response.content

        root = etree.fromstring(data)

        return jsonify([
            {
                "uid": topic.findtext("guid"),
                "updateDate": parse(topic.findtext("pubDate")).strftime("%Y-%m-%dT%H:%M:%S.%fZ"),  # "2016-04-10T00:00:00.0Z"
                "titleText": topic.findtext("title"),
                "mainText": sent_tokenize(BeautifulSoup(topic.findtext("description"), "lxml").text)[0],
                "redirectionUrl": topic.findtext("link")
            }
            for topic in root.xpath("/rss/channel/item")
        ])


if __name__ == '__main__':
    app.run(debug=True, use_reloader=True)

Note that, since there is a news description length limitation, I had to tokenize the descriptions and cut the first sentences only for the Alexa to read - for that, I've used sent_tokenize from nltk - not sure, if this is an overkill for such a straightforward problem.

Project is also available on github, hosted on Heroku.

\$\endgroup\$
3
\$\begingroup\$
  • You probably want to move the session to a global variable(or simply use requests.get) else at function level it is not helping in re-using any connections right now and is adding an extra level of indentation to your code.

  • This is mostly opinionated but grouping from x import y based imports below import z makes the imports bit easier to read. Plus keep the items within each group sorted as well. Google's style guide also contains a related example.

    import requests
    from bs4 import BeautifulSoup
    from dateutil.parser import parse
    from flask import Flask, jsonify
    from lxml import etree
    from nltk import sent_tokenize
    
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.