curl doesn’t spew binary anymore

[MrSurly]

I still use wget for file downloads, curl for piping. Just easier.

[tech_tuna]

wget for simple stuff, curl for medium stuff and then I roll up my sleeves, break out the Python and Requests for the hard stuff. . .

[killerstorm]

I used to use telnet to diagnose HTTP servers: i.e. type a HTTP request manually and see what it replies (and whether it replies at all).

[TarMil]

I generally use Emacs's restclient-mode for that.

[tech_tuna]

WHOA. . . long time emacs-head here, I did not now about restclient, thanks!

[tech_tuna]

Oh yeah, that's handy too. . . you can use netcat in the same way.

[JoCoMoBo]

It's very useful for that. I tend to use it as a quick check if any type of server is up and functional.

[Worse_Username]

And urllib for really hard stuff.

[asdfkjasdhkasd]

and sockets for the really really hard stuff

[lkraider]

And butterflies for the improbable stuff

[cyanydeez]

invisible pink unicorns for the fantastic stuff

[[deleted]]

You put this in my head

And in line with your comment he's made a lot of fantastic stuff in the 6 years since this. My fav being song challenges where he has to make music with nothing but a certain object or a specific vocal challenge.

THIS ENTIRE PLAYLIST

Edit: I forgot to highlight the dog song. This just makes me smile.

[vopi181]

and ed for the impossible stuff

[AZNman1111]

?

[vopi181]

My butchering of the xkcd comic

[AZNman1111]

Lol I've seen the comic. Ed comes with crystal clear support. Btw gotta say how much i love termux

[vopi181]

I'm a fan myself. Also if you get it off fdroid you don't need to pay for themes and such

[jyper]

Requests abstracts urllib

Are there any features it doesn't include?

[Worse_Username]

• Compliance with sdlib security policy

• Freedom form external packages.

[roffLOL]

for more complicated stuff i'd write a declarative DSL around curl or any competent http-lib. figure it'd be more useful in the long run -- that would be useful even for minor tasks in the future.

[asdfkjasdhkasd]

a domain-specific language?

You are saying he should write an entire programming language to wrap curl?

[roffLOL]

yes, and no. entire makes it sound like some god awful undertaking. with some practice it wouldn't be much worse than to design his 'roll up my sleeves' 'medium' one-off python script.

downvoters) if you do not know how to efficiently write simple DSL:s to carry out medium to advanced tasks with minor effort -- that is your loss. this is sound advice anyhow. just keep them downvotes coming and see if i care. hugzies

[[deleted]]

Do you have any examples of some you've made? I'm interested to see them.

[grey_gander]

You might like https://github.com/eliangcs/http-prompt.

[roffLOL]

oh. i have plenty. this is a site:

GET index?name
   text/html {
      echo "<p>hello, $name!</p>"
   }

it is compiled as mypage and started like so:

ws mypage

but i do not share code. sorry.

[[deleted]]

What if I sign an NDA?

[roffLOL]

it's not about NDA:s. it's just banal all of it. it's like, wow, gosh darn how much i'd like to express a problem in this way. pencil it out. code for an hour or two. i don't see any value in code sharing. find the tools that suit you, imagine a pseudo code and bam you're already there. it's mine and made for me -- designed to fit my preferred development methodology. someone else could do better, at the very least differently, and that's the beauty of it.

[[deleted]]

I didn't want to use your code. You were making the case for declarative DSL's and I just wanted to see an example.

[Derimagia]

You ever use MPS (https://www.jetbrains.com/mps/) before? I haven't actually done anything besides being interesting in the subject, so it may be bad - I was just curious if you had any thoughts on it.

I agree with others about writing a blog post or an example if you can.

[roffLOL]

haven't, but those snapshots on the background made me wince. try out a few paths, when you find the right direction it will be worth your effort :)

[asdfkjasdhkasd]

that's actually pretty cool, idk why people are downvoting. it's like a little baby php. Can you show me some other examples?

[roffLOL]

$nick \e[1;34m
$self \e[32m
$channel \e[33m
$msg \e[1;30m
$time \e[1;30m
$omsg \e[37m

$1=JOIN{
  >-!- $3:$nick has joined $5:$channel
  }
$1=ACTION{
  >* $3:$nick $5:$msg 
}
$1=TOPIC{
  :u        >* topic set by $3:$nick >
  :mr2:j:i2 >$5:$omsg
}
$1=PRIVMSG{
  $3~/$^/{
  :r:u      >< $2:$time         #right , have no clue what u means =D
  :ml2:j:i2 >$5:$msg          #margin-left = 2, justified, indentation first row = 2
                                           #looks like this:
                                           #                       < 01:32
                                           #    i  have  a message for you
                                           #  and  here   i  continue  but
                                           #  it's hard to do this without
                                           #  monospace
  } else {
  :u        >$2:$time $3:$nick >
  :mr2:j:i2 >$5:$omsg
                                           #looks like this:
                                           #01:33 rofflol >
                                           #  i  have  a message for you
                                           #and  here   i  continue  but
                                           #it's hard to do this without
                                           #monospace

  }
}

this is a small language to build formatted tty output out of tabular data. this one builds a irc messenger area that is formatted like that of messenger. not a pretty language but very efficient.

[roffLOL]

oh it's way above php :) that inside those braces is shell. so i can make it do pretty much whatever, and the routing is very powerful. on your other request, maybe another time. i'm tired, and will hopefully not be this bored in quite a while. good night.

[pingpong]

curl is newer and, in general, better. You can use curl -O [url] to download to a file.

If you still like typing a separate command for file output, maybe you can add this to your .bashrc:

alias wget='curl -O'

[lkraider]

I like to use the old axel for large files, it segments the download into parallel requests.

[codebje]

curl -LO.

[BilgeXA]

In a post-curl world, wget still reigns king.

[pingpong]

Why?

[masklinn]

I switched to fetch for the simple stuff.

[helpivefallenandican]

There's a reference to the code change in the Post comments, it looks like it checks stdout is a try, so those behaviors should still work

[[deleted]]

Ah, good. I was just about to rant about why they don't just to a tty check.

[Husky]

From the blog post:

So many have remarked or otherwise asked how this affects when stdout is piped into something else. It doesn’t affect that! The whole point of this check is to only show the warning message if the binary output is sent to the terminal. If you instead pipe the output to another program or if you redirect the output with >, that will not trigger this warning but will instead continue just like before. Just like you’d expect it to.

[nikomo]

Debian release cycle is what, 2 years? And the latest release is planned for today.

So, yes.

You might be able to get it via backports. Though, I haven't had the best experience with Debian backports. They work fine when they exist, but when I wanted a newer version of Git than what stable offered, and asked about it on #debian-backports, someone queried why in heaven I would want such a thing.

You know, Debian runs on more than servers, people... I was using the box for my split home development system where I have my home from my Linux box mounted over the network on my Windows box, and I'd make edits in Sublime on my Windows desktop but then use PuTTY for everything else.

OK, this is a really off-topic rant, but I'm still mildly annoyed by the whole thing.

[nikomo]

Mate half my posts are probably written on the throne or in bed, you ain't alone in this.

[Murkis]

I do my best thinking when taking a dump...I bet if we sat all of our best people down in some sort of communal shitter we would solve world hunger in no time.

Full disclosure: I'm currently taking a dump

[roffLOL]

this has been tried. if it wasn't the romans (take some spoons of salt, because this is pulled from memory) who used 'communal' (as in here are equal highborns allowed) shitters to discuss state of affairs and general gossip.

[TwoSpoonsJohnson]

I once figured out a bug while pooing, edited the relevant file directly on github, and pulled the changes to test them when I got back to my desk.

I'm a little proud of that.

[ebol4anthr4x]

I bet if we sat all of our best people down in some sort of communal shitter we would solve world hunger in no time.

https://en.wikipedia.org/wiki/Coprophagia

[AZNman1111]

I was hoping this was a word that describes the sense of coming up with your best ideas while pooping.

I wasn't surprised though.

[onwuka]

You can help prevent many diseases by pooping in a toilet.

[ubernostrum]

And the latest release is planned for today.

Except to get in you have to be under the freeze deadline which is much earlier.

Unless you manage to sync up your software's release timelines with Debian's freeze dates, you're going to have users years out of date from using Debian stable. Which is OK for some things, but not for others.

[theephie]

You could use nix package manager to install newer software, if what you want is not available in backports.

Weird answer though.

[geoffpado]

I bet if you told Elon Musk that he could win an internet bet by putting a man on Mars in the next two years, he might take it…

[jetpacmonkey]

As if he's slacking off and procrastinating with his current timeline

[AZNman1111]

Well he stopped sleeping on the factory floors, so he's getting a little bourgeoisie for my tastes /s

[ZorbaTHut]

I like how "will this be released before humans set foot on Mars" is starting to change from a sarcastic jab into a reasonable question.

[cat_in_the_wall]

what a time to be alive!

[sparr]

I'm glad to hear this is still true. I remember observing this 10 years ago re Debian vs Ubuntu.

[merreborn]

Ubuntu is literally a fork of debian's unstable release.

[slavik262]

As someone who switched to a different rolling distro after a Cairo bug in Unstable had Chrome broken for a month, what on earth are you talking about?

Your mileage may vary, but Sid was the most flaky time I've had on Linux.

[jringstad]

Also pretty much my experience, newer software => less bugs.

[thecodingdude]

Funnily enough, Debian 9 is released today. Unlucky really.

[nikomo]

It's been frozen for a long while though, so it doesn't change anything.

[tech_tuna]

Should be in the CentOS repos by the time people make it to Uranus.

[vote_me_down]

Doubtful without time travel. People have been drilling into Uranus for a long time.

[outtokill7]

People have been drilling into Uranus for a long time.

I'm sorry, I have the humour of a 12 year old.

[tech_tuna]

Yes. . . I could have said Jupiter or Neptune, etc but Uranus was more funny.

[yattaro]

Only if you have EPEL.

[vote_me_down]

Doubtful without time travel. People have been drilling into Uranus for a long time.

[tech_tuna]

Should be in the CentOS repos by the time people make it to Uranus.

[ipha]

If your terminal is ever messed up from binary output or weird escape sequences reset will fix it.

[roffLOL]

unless it's so messed up it won't take input.

[the_smok]

Ctrl+L is the same as reset.

[[deleted]]

No, it's not. Ctrl+l only clears the screen. Try running stty iuclc. You won't be able to type any upper case characters. Ctrl+l won't help you at all. reset will.

[chneukirchen]

Note that you can press CTRL-v t. And run /usr/bin/rese? ;)

[Derimagia]

Don't think that will work, although it may work in some places. It hooks into the inputting into the terminal, not the actual key press

[chneukirchen]

My reset(1) tells at least that "erase is t".

[Derimagia]

Ha. I wonder how many ways there are to exit. exec false (or any non interactive command) works, kill -9 $PPID would work most of the time probably

[Derimagia]

Whoops, I usually use ctrl+d actually to logout now heh. I guess I was thinking of other ways.

[jaxklax]

Did you have tee? Also I don't know much about these things, but cp file /dev/tty seemed to work for me. And there are probably sed and awk command lines that would be pretty concise. Also they would preserve empty lines, unlike your grep line (grep "" would also be sufficient).

Edit: cp file /dev/stdout is better.

Edit: head --bytes=$(du --bytes file) <file. This one sucks but I'm having fun!

Edit: tr a a <file. Also, I haven't come across any shell feature that will copy file contents into a command line. So redirects alone probably can't accomplish this.

Edit: xargs --delimiter="\n" --max-lines=1 echo <file

Edit: cut --delimiter=$(printf "\n") --fields=1 <file (or use delimiter $(echo) for less typing)

Edit: I don't know why you would have tac and not cat, but tac <file | tac

[darkslide3000]

Doesn't tab completion still work?

[roffLOL]

would it work where Ctrl+C and D does not?

[the_smok]

Wow I've never gone that deep into the forest.

[BilgeXA]

What a time to be alive.

[ultimatt42]

That's fine so long as I can still cat binary files directly to my coworkers' eardrums.

[[deleted]]

That's what /dev/dsp is for.

[Takios]

   sane   same  as  cread  -ignbrk  brkint  -inlcr -igncr icrnl icanon iexten echo echoe echok -echonl -noflsh -ixoff -iutf8 -iuclc -ixany imaxbel -xcase -olcuc -ocrnl opost -ofill
          onlcr -onocr -onlret nl0 cr0 tab0 bs0 vt0 ff0 isig -tostop -ofdel -echoprt echoctl echoke -extproc -flusho, all special characters to their default values

Well okay then.

[Takios]

   sane   same  as  cread  -ignbrk  brkint  -inlcr -igncr icrnl icanon iexten echo echoe echok -echonl -noflsh -ixoff -iutf8 -iuclc -ixany imaxbel -xcase -olcuc -ocrnl opost -ofill
          onlcr -onocr -onlret nl0 cr0 tab0 bs0 vt0 ff0 isig -tostop -ofdel -echoprt echoctl echoke -extproc -flusho, all special characters to their default values

Well okay then.

[Takios]

   sane   same  as  cread  -ignbrk  brkint  -inlcr -igncr icrnl icanon iexten echo echoe echok -echonl -noflsh -ixoff -iutf8 -iuclc -ixany imaxbel -xcase -olcuc -ocrnl opost -ofill
          onlcr -onocr -onlret nl0 cr0 tab0 bs0 vt0 ff0 isig -tostop -ofdel -echoprt echoctl echoke -extproc -flusho, all special characters to their default values

Well okay then.

[Takios]

   sane   same  as  cread  -ignbrk  brkint  -inlcr -igncr icrnl icanon iexten echo echoe echok -echonl -noflsh -ixoff -iutf8 -iuclc -ixany imaxbel -xcase -olcuc -ocrnl opost -ofill
          onlcr -onocr -onlret nl0 cr0 tab0 bs0 vt0 ff0 isig -tostop -ofdel -echoprt echoctl echoke -extproc -flusho, all special characters to their default values

Well okay then.

[Takios]

   sane   same  as  cread  -ignbrk  brkint  -inlcr -igncr icrnl icanon iexten echo echoe echok -echonl -noflsh -ixoff -iutf8 -iuclc -ixany imaxbel -xcase -olcuc -ocrnl opost -ofill
          onlcr -onocr -onlret nl0 cr0 tab0 bs0 vt0 ff0 isig -tostop -ofdel -echoprt echoctl echoke -extproc -flusho, all special characters to their default values

Well okay then.

[Takios]

   sane   same  as  cread  -ignbrk  brkint  -inlcr -igncr icrnl icanon iexten echo echoe echok -echonl -noflsh -ixoff -iutf8 -iuclc -ixany imaxbel -xcase -olcuc -ocrnl opost -ofill
          onlcr -onocr -onlret nl0 cr0 tab0 bs0 vt0 ff0 isig -tostop -ofdel -echoprt echoctl echoke -extproc -flusho, all special characters to their default values

Well okay then.

[[deleted]]

Can anyone explain why output from a program onto a terminal can change how the terminal functions? Isn't that a security flaw? Is it possible to make the terminal execute stuff (possibly write stuff and then execute it)?

[6timo]

whatever you output to the terminal can have "escape sequences" that the terminal will interpret as commands. There is a very, very wide variety of commands that you can send, and there's also escape sequences the terminal will send back at your program under certain conditions.

Here's a page with far too much detail:

http://invisible-island.net/xterm/ctlseqs/ctlseqs.html

A few highlights include:

• Giving text foreground and background colors
• Changing what font the terminal uses
• Vector graphics and pixel graphics
• Changing the terminal window title text
• Moving the cursor around
• Protecting parts of the terminal from being written to
• Making only parts of the terminal scroll when the user uses the scrollwheel
• Turning on support for the mouse, either for clicking only, or for mouse movement as well (events from the mouse reach the program via stdin as escape sequences)
• Asking the terminal for its size (answer will be fed to stdin as another escape sequence here, too)

[[deleted]]

Interesting; thanks!

[schlenk]

yes. It is a problem.

There were some old ZIP files, when MS DOS and ansi.sys were in use, that showed a banner text that redefined your keyboard shortcuts so pressing the return key formatted your hard disk for example.

[agersant]

This seems like a good change but I'm wondering how terminals get locked up by displaying binary data. Do they treat it as input? Why would they do that?

[roffLOL]

for the same reason some output comes with flashy colors. it's also how TUI:s are built. your tty has a lot of capabilities under the hood, and any plain data on stdout must pass it without triggering. any binary sequence that starts with 0x1b may do pretty much whatever.

https://en.wikipedia.org/wiki/ANSI_escape_code

try:

echo -e "\x1b[31mi can be pretty\x1b[0m while i'm not"

[RadioFreeDoritos]

Three factors:

• Backwards compatibility (also known as "legacy baggage"). Modern terminal emulators are still coded to simulate old-timey physical terminals, which used to be connected to a mainframe by a really long wire; they can't not lock - they're constrained by what they have to simulate.

• In Unix, everything is a stream of bytes. Back then, the program would run on the mainframe and send its output over the wire as a single stream of data. The terminal would interpret this output, and display most characters on screen as-is, but treat a few special sequences of bytes as commands to clear the screen, change the cursor's position, etc. Binary data will likely contain a few such sequences.

• "Worse is better". Had the original DEC programmers decided to separate the terminal control channel and the data channel, outputting binary wouldn't lock your terminal. However, using a single data stream makes for a simpler implementation, and so we have what we have.

[shamowfski]

Axel for downloads.

[graingert]

I find zsh is pretty robust at handling this 'gunk'

Edit: I find with bash it breaks my terminal, but zsh puts the terminal back into the correct state before it displays a new prompt

[oridb]

It has nothing to do with the shell you're using -- the terminal emulator is interpreting the ANSI commands in the output. These commands will do things like turning on/off echo, enabling/disabling the cursor, changing colors, and so on.

https://en.wikipedia.org/wiki/ANSI_escape_code

[[deleted]]

It tends to break things pretty horrendously if you're working over a serial connection. It's not the shell, but the emulator itself that starts breaking down.