To start, define a set of written scenarios that represent how typical users with average knowledge would use the program. You aren't looking to test every feature of the program, just the tasks that most users would want to do. Make your scenarios realistic; provide a short description of each scenario and ask the tester to perform tasks within that context. Write scenarios that don't lead the tester by using the product's words for actions, especially if your typical users of average knowledge might not know those words.
For example, if you want to evaluate an editor, your scenarios might ask the tester to enter a short text document, save it, and make basic edits to the file. For a web browser, your scenarios could ask the user to search for a website, bookmark a site, or save a copy of a web page for offline use.
Invite testers for a usability test. You don't need very many people, five testers will give you great results. Give the tester one scenario at a time, each on a separate piece of paper, and ask them to complete the tasks. Observe what they do in the program: the routes they take to accomplish the tasks, and the problems they encounter. Take plenty of notes.
The most difficult part of a usability test is watching a tester struggle to locate a menu or button. While the right action may seem apparent to you, the value is in learning and identifying what is not obvious for a user. Do not give hints. If a tester is unable to finish a scenario, that’s okay; just move on to the next scenario.
At the end, ask followup questions of your tester. For example, you might ask “You seemed lost when you tried to do X, what would have made it easier?” Or perhaps “What were you expecting to see on the screen when you were doing Y?” As a wrap-up, ask the tester to describe what worked well in the program, and what features should be improved.