3 years to apply for a refund 3 years from the time the taxes was filed?

Nope.

The agent had read a couple of paragraphs more than you let on here.

Sec 6511(b)

b) Limitation on allowance of credits and refunds

(1) Filing of claim within prescribed period

No credit or refund shall be allowed or made after the expiration of the period of limitation prescribed in subsection (a) for the filing of a claim for credit or refund, unless a claim for credit or refund is filed by the taxpayer within such period.

(2) Limit on amount of credit or refund

(A) Limit where claim filed within 3-year period

If the claim was filed by the taxpayer during the 3-year period prescribed in subsection (a), the amount of the credit or refund shall not exceed the portion of the tax paid within the period, immediately preceding the filing of the claim, equal to 3 years plus the period of any extension of time for filing the return. If the tax was required to be paid by means of a stamp, the amount of the credit or refund shall not exceed the portion of the tax paid within the 3 years immediately preceding the filing of the claim.

(B) Limit where claim not filed within 3-year period

If the claim was not filed within such 3-year period, the amount of the credit or refund shall not exceed the portion of the tax paid during the 2 years immediately preceding the filing of the claim.

(C) Limit if no claim filed

If no claim was filed, the credit or refund shall not exceed the amount which would be allowable under subparagraph (A) or (B), as the case may be, if claim was filed on the date the credit or refund is allowed.