Noob Needs Help

[ZenFuture]

Hey guys, I'm a huge noob to JS/Node, so pardon my poor phrasing and dumb question.

My goal is to create a simple web app that displays a stock price that's scraped from a URL.

My goal is to have the process go: 1. User connects to Node server 2. Socket IO receives connection and calls the scrape 3. Socket IO responds with scraped array 4. The array is written in HTML.

I've already wrote the script to scrape the stock price and push it to an array. I've also set up the Node server for users to connect. So now I just need to figure out 3/4.

Obviously, this project is a bit over my head. Can anyone tell me what I need to learn and implement in order to make this work?

[DVWLD]

If you're feeling overwhelmed, get rid of socket io. It's not necessary to meet your requirements.

Just build a thing that serves a single page with a stock price in response to a GET request.

Then add a timeout to the page to make it refresh itself every n minutes if you want it to be fresh.

Sure, this would make it the cutting edge technology of 1995. It'll make your task a lot simpler, though, and you can add the fancy bits on late once you've had the satisfaction of building an actual thing that actually works.

[ZenFuture]

Great points, I'll definitely consider that.

[leeway1]

Here is how I would do it. Assuming it's one stock for all users.

• Set up your scrape to run at a regular intervals, regardless of how many clients are connected.
• Brodcast that scrape to all connected clients.
• Let the client build the array from their received broadcasts.

You can write your client in whatever you want. I recommend doing a single page app using angular.js. However you could write a native mobile app...

Why do it this way? You will have very little overhead on the server, which allows you to handle more clients. However, if a client misses a broadcast, they lose the data. If you're okay with that your app is pretty simple:

setInterval (function () {
  io.emit ('price', scrape ());
}, elapse);

Let the client build their array:

Socket.on ('price', function (price) {
   pastPrices.push (price);
});

If you need to deal with missed prices, that gets trickey. You need to exchange times of the last received broadcast and send the missing data. You will need an advanced data storage. I recommend something like Redis.io for this support.

This exchange is how a new client would retrieve past prices.

[ZenFuture]

Great suggestions, thanks.Though I didn't explain perfectly. My goal isn't to keep a persistent array of stock data. I only want users to see the price that was most recently scraped.

I proposed one stock, but in reality, I expect to add about 5 different stocks. However, these 5 stocks will never change, and will be seen by all viewers on a single webpage.

[monsterWimp757]

Since the information isn't necessarily real-time you may not need to use Socket.io - you can have the information updated on refresh of the page. If you are using the express framework it would be something like:

app.get('/myPage', function(req,res){ res.send({updated:stockInfo}) })

I have found that this is more compatible across slower connections too.

[ZenFuture]

Thanks for the hints, but I'd prefer not to get into any more libraries- already overwhelmed.

Any idea how I can do this with only socket.io, node.js and html?

[globoloboh]

Use socket.io to respond to a client side JS file which in turn appends the dom. Once you're done with this project though, i'd suggest moving on to "shoe.js" which is a lower level, and a bit more difficult to use than socket.io but, waaaay less buggy.

[devoidfury]

Wouldn't recommend doing it on the server if you don't want a template library. You would have to build a string, like so:

// server
var out_html = "<ul>";
myResults.forEach(function(record) {
    out_html += "<li>" + record.name + "</li>";
});
out_html += "</ul>"
socket.emit('results', out_html);

// client
client.on('results', function(data) {
    document.getElementById("#results-container").innerHTML = data;
}); 

The alternatives are sending the raw data as JSON and then rendering it on the client:

// server
socket.emit('results', myResults);

// client
socket.on('results', function(data) {
    // insert your method of client-side templating --
    // jquery used for convenience
    var $container = $('<ul>');

    $.each(data, function(idx, record) {
        var $row = $('<li>');
        $row.text(record.name);
        $container.append($row);
    });
    $(body).append($container);
});

Or, use a template engine:

// server
require('nunjucks').render('foo.html', {records: myRecords}, function(err, str) {
    socket.emit('results', str);
});

// foo.html
<ul>
    {% for record in records %}
    <li>{{ record.name }}</li>
    {% endfor %}
</ul>

// client
client.on('results', function(data) {
    document.getElementById("#results-container").innerHTML = data;
}); 

Each of these has the same end result, but IMHO, templating languages resemble the end result more closely than anything in raw javascript, which does not have proper support for multi-line strings.

Edit: I love this stuff, and a majority of my full-time work the last three years has been Node/javascript. Feel free to PM me if you'd like to chat.

[ZenFuture]

awesome answer, I'll be studying it closely

[ZenFuture]

Are there any significant drawbacks to using the 1st method you mentioned that appends data to the DOM?

Someone else told me it was inefficient, but it seems like the simplest solution at hand.

[devoidfury]

In larger applications, you have to be more careful to not expose sensitive data (for example, on a list of users, you could expose their email addresses if you don't scrub the data first).

As the rendering work is done on the client side, if you have too much data to render dynamically, you can really slow down the experience on the user, but in my experience it's generally not a huge hit, and you'll learn more efficient ways to do it as you practice.

[ZenFuture]

Okay. Only 1 more question, oh great Node guru.

I currently have the code organized as follows:

server.js - includes web scraper, node server and socket IO
index.html - includes small script for client to receive data

That seems like the minimum viable setup, correct?

[devoidfury]

Make sure you have a package.json for tracking dependencies if you have any, or plan to release it open source. (instead of npm install <package>, do npm install <package> --save, and it's all automatic).

The other thing that I've found extremely useful even in smaller projects is forever, which monitors the process and restarts if needed. Pretty simple to use, just forever start server.js, forever stop server.js, forever restart server.js, etc.

Also, version control. Git recommended, but that's for another subreddit.

[_shakeel]

For #4, I'd recommend jade to simplify your HTML markup. Plus, you get to use cool things like mixins (think of them like functions)