This sort-of tutorial on quick and dirty mouseover effects uses two free programs, Mouse Over Button Wizard (MOBW) and the "Composer" application in the Mozilla Seamonkey all-in-one internet browser suite (based on the former Netscape Navigator).
You can download MOBW here, and Seamonkey here.
Both are absolutely free, and rather neet to have, if you like playing with things on your computer...
(I used Seamonkey to build this page.)
Okay.  So you have decided what two images you want to show for the "mouse off" and "mouse over" conditions.
For this demo, i'm using the Chickweed strips from 22 October 2003 and 22 October 2005 (my fifty-fifth and fifty-seventh birthdays).
The URLs at the HouChron for these are:
http://assets.comics.com/dyn/str_strip/000000000/00000000/0000000/200000/10000/4000/900/214920/214920.full.gif
and
http://assets.comics.com/dyn/str_strip/000000000/00000000/0000000/200000/10000/5000/600/215648/215648.full.gif
MOBW startup
Above you see the startup screen for Mouse Over Button Wizard (MOBW).

In this shot, i have pasted the link for 10/22/03 into MOBW, in the "Original Image Source" box.

Notice that it also (automatically) shows in the "OnMouseDown Image Source", as well.  You can put a different image's URL in here if you want;  if the "Original Image Source" is a picture of a button, you could, maybe, put the link here to an image of that button pressed down, so that clicking the button seems to actually press it.
{Click here for a demo.}

(The images used don't have to be online images - they can be images on your computer, like the button images and background files used here and in the button demo, but if you  use images like that, you have to be sure to upload them to the same server where you upload the HTML file for the page.)

URL for the mouse-over image pasted into the second blank.
(Something i haven't mentioned yet is that the two images need to be the same size, within a few pixels, at least.
If one is smaller than the other, it should be the second image; if the images are not exactly the same size, make sure you test the mouseover effect before posting it, to make sure that the difference in sizes isn't great enough to look bad.)

URL for the target page to go to when the button is clicked.
Notice the box labeled "Target".
The default is "_self", which will open the page in the same window or frame, replacing the current page.
I've changed that to "_blank", which will open a new page (or tab) for the new page, and leave the current page undisturbed.
This is strictly a matter of personal style and taste, so do whatever looks best to you.
You can also put something in the "Alternate Text" box; this is text that will appear as a link to the target page in place of the button if, for some reason, the browser is unable to display the images.

Okay, We're finished playing in MOBW.
Now we open up Seamonkey, click on "Window", select "Composer" and see something like the image above.

Click the button marked <HTML> Source, and you'll see the HTML source code for the current page.
Not much there yet - basically the <DOCTYPE>, <html></html>, <head></head>,
<title></title> and <body></body> tags

Here is the source HTML after we paste the part of the code generated by MOBW between the <head> and </head> tags, and type in the title we want to see in the browser's title bar when the page is displayed between the <title> and </title> tags.
{Click here to see this image full-sized.}

And here we see the code we have pasted in between the <body> and </body> tags.
{Click here for the full-size image.}

Save your Seamonkey file, close Seamonkey, go to the folder where you saved the file you made, double click on it so it opens in your browser, and you should see the dcreen above when it opens, and the one below when you move your mouse over the picture.
If so,
Congratulations!
If not, try reading the Seamonkey help files, or consulting an online HTML tutorial, like this one or this one.  (I haven't checked out these sites after i found them with Google, except to see that they do seem to be what they claim.  If you don't like them, Google on something like HTML beginner tutorial and you should find plenty.
{Click here to see the example we've been building actually work...}