Personally, I like my rear wiper.. it's useful as hell in the rain.. but that's besides the point here..
Mentioned earlier was to upgrade your exhaust. Unfortunately to most, that's not exactly the "least" expensive upgrade. There are quite a few things you could do prior to that, unless you're talking about JUST a muffler tip, which would then be inexpensive..
I would probably go with springs before exhaust, at least for modifications sake. Springs will run you about $200-$300 for decent ones, depending on where you get them, where decent exhausts will run you about $500'ish. This price is including installation (if you choose not to do it yourself)
If you have some kind of aftermarket rims (and not steelies with hubcaps), you can paint your calipers ($10 for a can of high-temp caliper paint).
You can get strut bars for your car. I got a set of Mugen replicas for $40 on ebay, brand new (and they're good).
You can get your windows tinted. I happened to find a place that warranties your tint for a lifetime, all windows excluding the moonroof for $180... using Llumar tints (good quality!).
Rear seat removal is a good one for minor weight reduction, and that's free!
These are just a few things you can work on in the meantime. Those prices are relatively adequate for what you could consider "mods," whether visually or for the sake of performance. I'd say most people would be able to perform those mods without having to dig too far into their savings, even if they do bust their asses at a restaurant. =)
Anyhow, welcome to Team-Integra. If you give this place a chance, this place will give you the world (for your car). hehe..