As you mentioned, when you bought it it only had the equivalent of 13,000 miles on it. The fluid should not be brown with such low mileage. It sounds like bad news. Either it was subject to heat degradation, and/or the previous owner beat the crap out of driving it.
I wouldn't bother to check the filter, as you probably won't find anything. All you can do is keep changing the fluid and hope for the best.