There are two possibilities if you're seeing a white background: Either the painting has white paint in the background which would need to be erased, or the canvas has not been made transparent.
The first thing to try is to open the canvas settings panel from the View menu and set Opacity to 0%. This means that the Paper that sits underneath your painting becomes transparent.
When you look at the painting now, the background area should contain checkers (which represent the fact that the canvas is see-through). If you don't see checkers then there is some paint sitting on the canvas causing the white background and you will need to locate it in your layers and delete it.
When you export as PNG with the canvas opacity set to 0% you should see transparency where there is no paint.
Hope that helps!