What exactly is a GPU?

The evolution of the GPU has gone from a "device for rendering images at high speed" to "a device for parallel processing of ...