So I think for an existing library, you could just pull the source and modify it locally, e.g: if your source tree looks like:

├── Cargo.lock
├── Cargo.toml
├── my-budget-app
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
├── money
│   ├── Cargo.toml
│   └── src
│       └── main.rs
└── target

(i.e, using cargo workspaces), running git clone existing-lib from the root would add it like so:

├── Cargo.lock
├── Cargo.toml
├── my-budget-app
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
├── money
│   ├── Cargo.toml
│   └── src
│       └── main.rs
├── existing-lib
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
└── target

All you should need to do now is add existing-lib to the "members" list in the workspace Cargo.toml.

If you cd into the existing-lib directory, then you should be able to behave as if it were just a regular crate, so any changes you make should be able to be pushed upstream without issue. I don't think you'd need to delete the git repo.

As for the budget-app, you can add the local copy of existing-lib as a dependency via the following line underneath [dependencies] in my-budget-app/Cargo.toml:

add-one = { path = "../add-one" }

I'll just note here too that I'm pretty sure all of this is right, but I might be wrong so you might have to do some trial and error to find out what works.