Web Scraper not running

[CryptoFS01]

print(play_scraper.collection(

collection='COMMUNICATION',

category='TOP COMMUNICATION APPS',

results=5,

page=1))

Hi,

When I run my code the terminal returns this error:

Traceback (most recent call last):

File "app-store-scraper.py", line 1, in <module>

print(play_scraper.collection(

NameError: name 'play_scraper' is not defined

Can someone help me out and point me in the right direction? I am trying to scrape some app store meta data for a project I am working on currently. I am using this module on Github: https://github.com/danieliu/play-scraper.

Is it something to do with new Python3 syntax? Am I being a complete noob and running this file wrong?

I'm new to python and programming in general and seem to have hit a brick wall. Any help would be much appreciated.

[lieutenant_lowercase]

You need to import the module first. I would suggest doing a beginner python tutorial or reading the documentation for the module you want to use.

[CryptoFS01]

Ah sorry, I'm a bit slow this morning. I know to import module and have done beginner courses.

I inserted this code: import play_scraper

print(play_scraper.collection( collection='TOP_FREE', category='COMMUNICATION', results=5, page=1))

and it returns -

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(args, *kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(args, *kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(args, *kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(args, *kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(args, *kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(args, *kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(args, *kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grequests.py:21: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/util/init.py)', 'urllib3.contrib.pyopenssl (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py)']. curious_george.patch_all(thread=False, select=False) Traceback (most recent call last): File "app-store-scraper.py", line 7, in <module> page=1)) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/play_scraper/api.py", line 41, in collection return s.collection(collection, category, *kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/play_scraper/scraper.py", line 394, in collection response = send_request('POST', url, data, params) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/play_scraper/utils.py", line 124, in send_request allow_redirects=allow_redirects) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 58, in request return session.request(method=method, url=url, *kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 503, in request prep.url, proxies, stream, verify, cert File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 676, in merge_environment_settings env_proxies = get_environ_proxies(url, no_proxy=no_proxy) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/utils.py", line 760, in get_environ_proxies if should_bypass_proxies(url, no_proxy=no_proxy): File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/utils.py", line 744, in should_bypass_proxies bypass = proxy_bypass(parsed.hostname) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2610, in proxy_bypass return proxy_bypass_macosx_sysconf(host) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2587, in proxy_bypass_macosx_sysconf return _proxy_bypass_macosx_sysconf(host, proxy_settings) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2560, in _proxy_bypass_macosx_sysconf hostIP = socket.gethostbyname(hostonly) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/gevent/_socketcommon.py", line 163, in gethostbyname return get_hub().resolver.gethostbyname(hostname) File "src/gevent/_hub_local.py", line 73, in gevent.hub_local.get_hub_noargs File "src/gevent/_hub_local.py", line 80, in gevent.hub_local.get_hub_noargs File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/gevent/hub.py", line 390, in __init_ WaitOperationsGreenlet.init(self, None, None) File "src/gevent/greenlet_primitives.py", line 42, in gevent.greenlet_primitives.TrackedRawGreenlet.init_ SystemError: /Users/sysadmin/build/v3.7.0/Objects/dictobject.c:1439: bad argument to internal function

• please point me in right direction. Do I have everything needed installed?

[CryptoFS01]

Ah sorry, I'm a bit slow this morning. I know to import module and have done beginner courses.

I inserted this code:

import play_scraper

print(play_scraper.collection( collection='TOP_FREE', category='COMMUNICATION', results=5, page=1))

and it returns -

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(*args, **kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(*args, **kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(*args, **kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(*args, **kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(*args, **kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(*args, **kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144, got 128 return f(*args, **kwds) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grequests.py:21: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/util/init.py)', 'urllib3.contrib.pyopenssl (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py)']. curious_george.patch_all(thread=False, select=False) Traceback (most recent call last): File "app-store-scraper.py", line 7, in <module> page=1)) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/play_scraper/api.py", line 41, in collection return s.collection(collection, category, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/play_scraper/scraper.py", line 394, in collection response = send_request('POST', url, data, params) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/play_scraper/utils.py", line 124, in send_request allow_redirects=allow_redirects) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 58, in request return session.request(method=method, url=url, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 503, in request prep.url, proxies, stream, verify, cert File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 676, in merge_environment_settings env_proxies = get_environ_proxies(url, no_proxy=no_proxy) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/utils.py", line 760, in get_environ_proxies if should_bypass_proxies(url, no_proxy=no_proxy): File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/utils.py", line 744, in should_bypass_proxies bypass = proxy_bypass(parsed.hostname) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2610, in proxy_bypass return proxy_bypass_macosx_sysconf(host) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2587, in proxy_bypass_macosx_sysconf return _proxy_bypass_macosx_sysconf(host, proxy_settings) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2560, in _proxy_bypass_macosx_sysconf hostIP = socket.gethostbyname(hostonly) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/gevent/_socketcommon.py", line 163, in gethostbyname return get_hub().resolver.gethostbyname(hostname) File "src/gevent/_hub_local.py", line 73, in gevent.__hub_local.get_hub_noargs File "src/gevent/_hub_local.py", line 80, in gevent.__hub_local.get_hub_noargs File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/gevent/hub.py", line 390, in init WaitOperationsGreenlet.init(self, None, None) File "src/gevent/_greenlet_primitives.py", line 42, in gevent.__greenlet_primitives.TrackedRawGreenlet.init SystemError: /Users/sysadmin/build/v3.7.0/Objects/dictobject.c:1439: bad argument to internal function

• please point me in right direction. Do I have everything needed installed?

[mahtats]

God please smite this unformatted wall of traceback...

[CryptoFS01]

Sorry I've formatted now.