The delay has to do with the buffer size of your audio card.  Any and all plugins you run from this same host is going to have this latency without some adjustments on your part.

In your plugin host, choose an audio driver that allows you to change the buffer size.  ASIO is good if available.

Smaller buffer sizes decrease latency.  Larger buffer sizes give the PC more time to process before a signal is due at the audio card.

If you don't get a driver option in your host(s) that allow changing the buffer sizes, then you could try something like ASIO4All.  If that doesn't work, then get a nice pro audio interface with ASIO drivers that is meant to be low latency and add that to your system.

Thanks.  I will experiment with the buffer.   I was not able to get it to list my external USB TASCAM (which supports ASIO2) box as an output, so I will try again with that also.

That improved it a lot, but not to a point of being able to play without going crazy.
Windows 7 Pro SP1 64-bit, Intel Core 2 Quad, 2.66GHz, 4.00GB RAM
TASCAM US-122L, Control Panel set to "lowest latency"  (Spec sheet:  D/A Conv (AK4384) 24-bit/96kHz, 19.3 samples delay, USB 2.0, Protocols ASIO, ASIO2, GSIF2, WDM)
Keyboard connected via MIDI to TASCAM.  Was worse with USB direct to PC
Garritan ARIA Player x64 v1.872, Engine v1.872, Sample Rate set to 96000 Hz (highest supported choice)  Buffer Size set to 4096 (lowest choice)

What changes would give me the most reduction in latency?

Try 44.1khz or 48khz clock rates.  Unless you need to do really detailed recording with very high end microphones, there is no good reason I can think of to need a clock rate higher than 48khz.  48khz is perfect if you go straight to video, or other online streaming formats.  44.1khz is great if you're going to CD, or happen to use any spdif gear that'll only work in 44.1khz.

24bit should be good for the dynamic range.

4096 is still a rather large buffer.... 

You should be fine with something between 128kb and 1024kb on that rig unless you're running quite a lot of plugins and/or effects (large project). 

The only time you should ever need audio buffers that large is when working with larger DAW projects that need extra cpu time to work without artifacts/pops/clicks/etc (Typically if you pile on lots of VST effects and a CPU core for that thread gets maxed out), and real-time input isn't important.

What driver choices do you see? 

Does your US-122L have an ASIO option?  If no ASIO, check that you have the latest drivers installed.

If there is no ASIO driver available for your Tascam, you can give the ASIO4ALL wdm to ASIO bridge a try.

What are you using as a host?

Here's an example of using the stand alone version of ARIA on my system with a Tascam US-1200 USB audo interface.

I.  Launch "C:\Program Files\Garritan\ARIA Player\ARIA Player x64.exe".

2.  In the top menu:  Tools\Preferrences I make the following choices:

Here you can see how I've set it in Finale.

1.  In the top menu:  MIDI/Audio/Device Setup/Audio Setup...

Note that with some TASCAM cards, you might have to first make buffer settings in its own control panel before it will show up as an option in some hosts.

Here is another example where I'm using ASIO4ALL and my built in sound-card (Which does NOT have an ASIO driver) in Finale.

Finally, here's an example of using ASIO4ALL to aggregate 3 audio devices for Cubase 9. 

It's using 48k sample rate, and 512k buffer going into ASIO4ALL.  In this case I wanted to run different monitors over different audio interfaces and easily be able to swap speakers to get an idea of how the mix sounds through various sets of speakers or headphones.  Note, when aggregating sound-cards like this they are not in true sync to the same clock, but this is fine in my case since I'm only ever using one of the cards at a time to listen to my mix.  This kind of ASIO4ALL aggregation also comes in handy to bring inputs into the DAW for things like stand alone USB mics or headsets.

There's actually no point in using anything other than 44.1kHz, 16 bit, as the samples themselves are all recorded at this CD quality.

This gives audio to 20kHz with something like a 96dB dynamic range. This is quite sufficient for live playing.

(If you're an organist, then the GPO5 library has a 'Custom Organ' in it which allows the tweaking of a very wide range of organ stops to your own satisfaction. )

Here's a screen shot.

Good point John, particularly with live gig setups.  44.1k projects at 16bit are good.  Most VSTi plugins and their libraries out there were most likely built with 44.1k samples at 16bit, so somewhere in the pipeline it gets resampled to match your project settings.

48k DAW projects at 24bit are also really common, simply because we're often mastering towards that goal.  It's very common for video, as well as a lot of the lossy compressed streaming encoders. 48k is getting to be a very common standard for DAW projects......

The really high clock rates at 24bit or higher are mainly of interest when RECORDING into the DAW with really high end mics and preamps capable of capturing transients that few speakers can translate, and even few human ears can detect anyway.  All of that extra sampling information can make a difference in audible ranges....but not with the sort of consumer range gear most of us have at hand :) 

Some cheap audio chipsets do sound a little better if we crank up the clock rate and upsample the live stream later on, but there's not really any advantage to recording at those rates with most of the mics and preamps on the market (and the disadvantage is MUCH bigger files on the hard-drive).

Thank you for all the thorough and detailed responses.  I think we might have drifted into a different topic than I was trying to discuss.  I am not recording.  Rather, I just want to plug the keyboard and the US-122L into the computer and just use ARIA Player to play sounds out of the US-122L's Line Out connections into an amp.   My keyboard has weak organ patches, so I want to substitute Garritan patches via ARIA Player.

Sorry Brian, but it's not 'topic drift'. 

You indicated you have a piece of hardware set to absurdly high clock rates, with massive audio buffers, on a Core Duo System  Architecture.  There are only a couple of sentences (mainly partial sentences) about recording in the entire thread, and each of those explain that you will NOT need clock rates other than 44.1k or 48k, or large buffers, UNLESS you wish to record with really good gear and use lots of plugins and/or processor intensive effects.

It is an explanation of when and why you might choose the different clock rates your TASCAM offers.

At any rate, if you wish to get rid of the latency, lower the buffer sizes and use a lower clock rate.  Personally I'd use 24bit mode, but 16bit is fine for your live performing scenario.

The only reason I threw the Cubase Hosted Setup in was to show that it's also possible to get ASIO support for almost any audio device via the ASIO4ALL driver bridge, with the added benifit of being able to aggregate ALL of your audio interfaces and use them together should the need arise. 

Plus, I gathered that the Tascam Interface did come with a version of Cubase LE, which would more than likely make a better host (flexibility in setup and control) for 'live playing' with ARIA than Finale.  In fact, I could teach you how to emulate on/off style organ stops in Cubase if your MIDI keyboard has enough buttons and such (I.E. a bank of MPC Pads).  In essence, using the HOST to emulate many of the properties of that Organ console John posted.

I don't know if you have Cubase installed and use it, but I'm pretty sure there is a high chance you have a Cubase LE Key (provided you got your Tascam New, and not second-hand).

Thank you, all, for the excellent advice.  When I revisited ARIA player, I saw that I had left it on MME which only allowed the buffer to go down to 4096.  When I set it to ASIO (as you repeatedly suggested...sorry for not catching it sooner), the buffer size dropped to 192 and problem solved!   :-)      Thank you thank you thank you!