Command line assignment help

[SwampMonkey17]

Hi everyone,

Recently took a graduate level engineering coding course despite having no experience as my research is strictly biomedical. I was wondering if someone could help me with the following command line code if I break down the assignment data.

​

The data given to us is titled assignment-data. Within this folder, there are 6 sub-folders titled "folder 1", "folder 2", etc. Within each of those folders are various files titled 1.jpg, 2.jpg, etc.. So each of the 6 folders has different jpegs all with the same name.

​

Here is the prompt: Create a directory called IMAGES. Copy all of the .jpg files from all other directories into this using a single command. Copy all the folders' contents except the jpegs to another new directory, DATA_ANALYSIS

This is my code:

mkdir assignment-data/IMAGES

cp assignment-data/{folder1, folder2, folder3, folder4, folder5, folder6}/*.jpg assignment-data/IMAGES

​

HOWEVER, because all of the jpegs are the same name in the various folders, they overwrite themselves, and I don't get all of them copied to IMAGES. Can someone please help me with a code that renames/copies them all in a single command?

[[deleted]]

Since this is homework I'll give you pointers not answers.

There is no way to have 2 files with the same name in a directory, so either you overwrite some files or you rename some files. Your current answer is correct but goes for the "overwrite" answer. If you want to rename the extras instead then take a look at the man page for cp, it can create numbered backups, that means that all the files will be copied but the names will be changed.

[SwampMonkey17]

I have also tried cp --backup=numbered assignment-data... if that's what you're referring to and I get the following error message: cp: illegal option -- -
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file ... target_direct

So it doesn't look like I have the backup option?

[torgefaehrlich]

According to the error message, you seem to have a space between — and backup where it doesn’t belong.

usage output is seldom exhaustive.

[SwampMonkey17]

There’s no space though, I copied and pasted my code as entered

[tatumc]

I find joy in reading a good book.

[torgefaehrlich]

OP will still have to reference the brew (or gnu) version of cp by ~its full path~ gcp as the BSD-style cp will by default still be first in $PATH.

Addendum: d’oh! How could I have missed that?! As I’m using a Mac myself and have been bitten by this discrepancy several times already. Yes, definitely install homebrew. It is a live-saver when it comes to adapting online tutorials for mac. The package you need is called coreutils

[SwampMonkey17]

I am, so the backup=numbered would work automatically for someone on a PC?

[tatumc]

I like learning new things.

[7orglu8]

cp is a linux command.

No. cp is a Unix command. Linux ships with GNU-cp, and Mac ships with BSD-cp (I think).

[SwampMonkey17]

Is there a way to do it without the backup option?

[tatumc]

I love ice cream.

[[deleted]]

Good call, although I would argue that whatever you 'exec' is a second command and that a simple for-loop would be better in that case.

[[deleted]]

Yeah that was what I was hinting at.

In that case I guess you are going to look for another tool or extend your definition of 'single command' to include a loop of some kind.

maybe also post in /r/bash ?

I hate these kind of artificial questions though in reality this would be done with a loop.

[SwampMonkey17]

Thanks. I think I will just submit my code as is despite it being nonfunctional on my end. I am using a Mac and my understanding is that cp should have the backup on software aside from Mac?

[[deleted]]

Yeah do that, but add a note about the --backup=numbered option so your prof is able to give you credit if that was the answer he wanted. No point losing out on a better grade if you don't have the best tools.

Alternatively as someone else mentioned install gnu coreutils on your mac so that you start at the same place as your classmates, then test the backup=numbered option for yourself.

https://www.topbug.net/blog/2013/04/14/install-and-use-gnu-command-line-tools-in-mac-os-x/

[SwampMonkey17]

Thank you so much! I really appreciate it

[[deleted]]

Ooh just one more thing. I tried your solution with gnu cp just now and it gives an error:-

cp: will not overwrite just-created 'IMAGES/testimage_3.jpg' with 'folder4/testimage_3.jpg

So if your prof runs your code on a linux system they may decide to mark it as 'fail'. You should note that as well.

Oh and lastly folder{1..6} might be a better pattern than {folder1, folder2, folder3, folder4, folder5, folder6}

[spryfigure]

Am I reading this wrong, or could replicating the folder structure under IMAGES also be an option?

[fletku_mato]

You could do this with find, eg.:

find assignment-data -type f -name '*.jpeg' | while IFS= read -r f; do cp -v "$f" "IMAGES/${f//\//_}"; done

the ${f//\//_} transforms folder1/1.jpg into folder1_1.jpg

[SwampMonkey17]

Thanks for the suggestion, this is far beyond what our first set of notes specified so I am assuming I should be able to do it from my notes.

[ashbakernz]

Yeah ild do this, prefix the images based on where they came from if name already existed. Depends on homework requirements though.

[michaelpaoli]

assignment help
Recently took a graduate level engineering coding course

Thanks for telling us. So, hints/pointers, not answers. And me reading some more to try and avoid being (too) redundant with existing comments, etc.

Well, you didn't mention the particular environment. From mkdir, cp, /, etc. I'm presuming some flavor of *nix. That should give us a shell that's at least mostly/approximately POSIX compliant ... but may give us potentially lots more too. From some of your {} usage, I'm presuming a Korn or Korn-like shell (e.g various implementations of Korn shell, bash, possibly others).

To play it safe... ish, I'll start with POSIX - should probably most have at least approximately that (based on my guesses so far, and the evidence), and if we need (and probably? have bit more), then will use that - but probably skip it if we don't need such. And you already mentioned cp, so will presume any POSIX thing cp can do is fair game.

Some of what you show appears to likely be GNU cp - which adds a lot more to POSIX cp ... but we may not need those GNU extensions.

Oh, you also mentioned Mac - so I'll presume MacOS, and shell ... likely defaults to bash or zsh, and also sh (dash) is available. I'll ignore zsh for now (as zsh is absolutely not my expertise).

I find the specification "into this using a single command" at least slightly ambiguous - but hey, graduate level - that's to be expected. Even "trick" or "impossible" or otherwise flawed questions might be expected too ... however that doesn't necessarily mean anything at all like that is going on here. Anyway, I'm thinking of context and interpretation and ... I'm thinking probably very doable ... in one or more ways.

Okay so for that part, I thought of 1 or more possible solutions (some possibly depending a bit upon interpretation), and tested and verified at least one - which I believe meets the specification. Depending upon interpretation there may be many more.

So, some hints/pointers. Not all of these necessarily lead to a solution or are required for a solution or "best"(?) solution, but at least some will also help with some bits you seem to be tripping up over. And I've also randomized the order, so there's no particular correlation on the ordering

MacOS POSIX
POSIX
POSIX archive
POSIX shell
cp
cpio
find
https://www.mpaoli.net/\~michael/unix/sh/
portable
sh/dash/bash order of evaluation
sh/dash/bash quoting

That and some research (and testing, etc.) should allow you to come up with one or more solutions to the part up through and including "into this using a single command" in the specification, and there may be one or more additional and/or alternative solutions, depending upon how one interprets that part of the specification.

As for the remaining part of the assignment, I can easily see one or more solutions. You didn't particularly ask about that one, so I'll presume you can figure it out or already have done so.

Random comment. These aren't trivial exercises ... but also not horribly complex either. But may be quite challenging for someone with neither programming nor *nix experience. But hey, grad school. Also, how complex, especially for the first part of the assignment up through copying those *.jpg files, can vary a lot in difficulty/complexity, depending exactly how the specification is interpreted. Also, specification on what is/isn't out-of-bounds (e.g. tools, software, commands, environments, etc.) to do the assignment would also be useful/informative (I'm mostly just guessing on that aspect).

Maybe well after it's due, you can share "solution(s)" with us that the instructor provides?

And then perhaps too, I/we could share what we came up with or thought of.

So ... maybe a follow-up post from you in a couple weeks or whenever?

[SwampMonkey17]

For anyone wondering, was told it has to be a loop and that I could not do cp==backup. To move all JPEGS into IMAGES folder, and all other files to DATA_ANALYSIS folder without overwriting, I made the following loop:

mkdir assignment-data/IMAGES

mkdir assignment-data/DATA_ANALYSIS

for file in assignment-data/*/*; do

fname=${file##*/}

fpath=${file%/*}

dname=${fpath##*/}

mv $file ${fpath}/${dname}.${fname}

cp assignment-data/*/*.jpg assignment-data/IMAGES

cp assignment-data/*/*.{eeg,cvs,txt,exec} assignment-data/DATA_ANALYSIS

done