Most references say that, like "towards," "backwards" is standard in Britain and "backward" is standard in America. The exception is that when you are using "backward" as an adjective, as in "her grandmother's backward ways" or "the program has backward compatibility," then you never use the "s." It is always "backward" as an adjective.

If you are in the U.S., you have it easier because you can just remember that it's always "backward" without the "s." We like shortcuts here, such as eating dinner in our cars, so you can remember that we've lopped off the "s." But if you are using British English, you have to remember that it's "backwards" as an adverb and "backward" as an adjective.