You're on the right track. If the fluid is truly clean (i.e. replaced recently), then its probably your master
Luckily it's a pretty easy DIY fix and cheap if you pay someone. Options are a straight OEM replacement (~$100), a rebuild kit (~$18), or the new RJM heavy duty master (~$300).
I just recently installed the RJM master for piece of mind. I had to replace my OEM master 3 times before the RJM master was even available.