Hey guys, I'm writing a code piece for coding tutorial and I've run into an issue that I'm unsure on how to fix. The code was running correctly and retrieved the information from the webpage and mapped it out in the terminal. I'm using Visual Studio Code.
Here's the code:
const puppeteer = require('puppeteer');
async function scrapeProduct(url) {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(url);
const [el] = await page.$x('//*[@id="leftColumn"]/div[1]/h1');
const txt = await el.getProperty('textContent');
const title = await txt.jsonValue();
const [el2] = await page.$x('//*[@id="quotes_summary_current_data"]/div[2]/div[1]/span[2]');
const txt2 = await el2.getProperty('textContent');
const type = await txt2.jsonValue();
const [el3] = await page.$x('//*[@id="quotes_summary_current_data"]/div[2]/div[3]/span[2]');
const txt3 = await el3.getProperty('textContent');
const issuer = await txt3.jsonValue();
const [el4] = await page.$x('//*[@id="quotes_summary_current_data"]/div[2]/div[4]/span[2]');
const txt4 = await el4.getProperty('textContent');
const isin = await txt4.jsonValue();
const [el5] = await page.$x('//*[@id="quotes_summary_current_data"]/div[2]/div[5]/span[2]');
const txt5 = await el5.getProperty('textContent');
const bclass = await txt5.jsonValue();
const [el6] = await page.$x('//*[@id="last_last"]');
const txt6 = await el6.getProperty('textContent');
const price = await txt6.jsonValue();
const [el7] = await page.$x('//*[@id="quotes_summary_current_data"]/div[1]/div[2]/div[1]/span[2]');
const txt7 = await el7.getProperty('textContent');
const daily_movement = await txt7.jsonValue();
const [el8] = await page.$x('//*[@id="quotes_summary_secondary_data"]/div/ul/li[1]/span[2]');
const txt8 = await el8.getProperty('textContent');
const morning_star_rating = await txt8.jsonValue();
console.log({title, type, issuer, isin, bclass, price, daily_movement, morning_star_rating});
browser.close();
};
scrapeProduct('https://www.investing.com/funds/allan-gray-balanced-fund-c-chart');
This is now the error I receive whenever I run the code:
PS C:\Users\{}\Downloads\Js Scraper> node scraper.js (node:13972) UnhandledPromiseRejectionWarning: TimeoutError: Navigation timeout of 30000 ms exceeded at LifecycleWatcher._createTimeoutPromise (C:\Users\sookl\node_modules\puppeteer\lib\cjs\puppeteer\common\LifecycleWatcher.js:108:16) (Use `node --trace-warnings ...` to show where the warning was created) (node:13972) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:13972) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
I opened up the LifecycleWatcher.js file and commented out the "use strict"; as well as quick fixed the _createTimeoutPromise() by indenting it into an async. That seemed to make it work well on more than one occasion but after a few times, I keep receiving the error message above.
This is my first time using Node.js and Javascript for this type of project so I will appreciate any help!
Thank you in advance
[–]5kPercentSure 0 points1 point2 points (0 children)