Below assumes you are a 501(c)(3):
A key question is if you had more than $25K in gross receipts during the tax years in question. If less than 25K, prior to the 990-N, you didn't need to file anything. If more, then the 990EZ was probably due. (If you didn't file, the IRS assumes less than $25K).
The 999-N (e-postcard) was introduced last year for the less than $25K group. You'll have to look at the timing on that one.
Filing Requirements
If you were less than 25K, you are fine. Either nothing was due, or depending on dates, the e-postcard. And there's no penalty for filing that late.
If the 990EZ is late, the penalty is $20 PER DAY with a maximum of $10,000, or 5 percent of the organization's gross receipts, whichever is less.
The penalty can be waived for "just cause". If you plead ignorance, inept treasurer, didn't know, just a poor little organization, remorseful-never do it again, you can probably negotiate out of the penalty with the IRS. This will take some correspondence back and forth and several months.
Normally I consider myself a pretty straight up, follow-the-rules kind of person. But here's where I find reality and practicality sway my thoughts on the matter. If no tax was due, no one is deprived of funds, no ill-intent, we operated properly in other aspects - I'd be hard-pressed to REPORT MYSELF only to spend the next 6 months negotiating out of a penalty the IRS will most likely waive. (Remember, the 990EZ is only informational. Unless you are in a very unusual situation, you owed no taxes.)
It's extremely unlikely the IRS will ever come after you. They really aren't trying to squeeze small organizations for penalty $$. They just want them to operate properly. Do the math. You've probably maxed out on the fine. In the highly unlikely event it comes up a couple years from now, if you've maxed out - it can't increase.
Good luck.
P.S. Let me be very clear this is strictly my personal opinion. You do what you feel is right for you!