Web crawl articles from currents API

Currents API, has few advantage over other public news APIs

1. It aggregates data from multiple sources
2. It aggregates data in different language and region
3. You can access article contents in paid version.

You will need API key before you can call this API, and you can register here.

Once you API key, you can either use API key directly or use environ variables. You might need to restart your system to call upon environ variable you just saved.

There are 2 APIs services available with currents —

1. Search — https://api.currentsapi.services/v1/search?
2. latest- news — https://api.currentsapi.services/v1/latest-news?

You can filter news articles on language, category, country, type, domain, keywords and start date.

Type — # 1 for news, 2 for article and 3 for discussion content

Domain — source of news for e.g. zdnet.com

You will start with importing libraries

import os
import requests
import pprint
import json
url = ('https://api.currentsapi.services/v1/latest-news?'
        'language=us&'
        'apiKey=aL3Ll_WX-OyjMY5BJSopz2gWPVjCzS1iB4aGGNCsVA8TueKi')
response = requests.get(url)
print(response.json())

‘* if there is no connection between strings, they are just added, like above.

‘* response.json() Returns a JSON object of the result (if the result was written in JSON format, if not it raises an error)

for some reason, it returned error for me, ‘’module’ object is not callable’. So I checked further into response object. Not sure why but had error with using pprint and no errors with print command.

r.raise_for_status()
print(r.headers)
print(r.text) - 'dict' object has no attribute 'text'

Once, I knew what the response object was, I was able to break it down in key value pairs.

r = requests.get(query_url)
print(type(r)) - <class 'requests.models.Response'>
r = requests.get(query_url).json()
print(type(r))
<class 'dict'>

For reading ‘dict’ object

r_dict = json.loads(r.text)
for i in r_dict:
    print("key:",i, "val:",r_dict[i])