When TCU remained No. 20 in both The Associated Press and USA Today's coaches' polls two weeks ago, Gary Patterson was surprised. He didn't think the Horned Frogs deserved the ranking after their win against Louisiana-Monroe, in which they trailed after the first quarter.There wasn't much change in his view after the TCU offense failed to score in the first quarter against FCS team Portland State the following week, but the Frogs, who have been ranked in the AP poll for 46 consecutive weeks, remained 20th in both polls.Not anymore. The loss to SMU in overtime dropped the Frogs out of both polls for the first time since the week before they upset No. 8 BYU 32-7 on Oct. 16, 2008. Since then TCU had remained a major player in the polls by going 29-2 through the 2010 season. Now, as the Frogs sit at 3-2, the reality of this season, one in which they were picked to finish third or fourth in the Mountain West by some media members, is changing.And that's despite the fact that with a few plays made in two games, the Frogs could easily be 5-0. But they'll be tested again at San Diego State (3-1) at 9:30 p.m. Saturday. Quarterback Ryan Lindley and running back Ronnie Hillman make up the best offensive one-two punch in the conference. TCU, though, dominated Air Force on the road in Week 2, which makes this a hard game to read."We probably played better and with more detail than we did in the two wins before," Patterson said. "The bottom line is we played a better football team."The defense remains in the spotlight because of its penchant of giving up the deep pass, but it's the TCU offense that has been a no-show early in three consecutive games. Patterson doesn't believe a conservative game plan has been the problem, pointing out that quarterback Casey Pachall went deep to Josh Boyce on the first series Saturday."We had a lot of young players make a lot of mistakes," Patterson said. "We haven't started really fast since Air Force. It wasn't conservative. You've got to give some credit to SMU."Defense improves?It's not too often Gary Patterson gives up 40 points and finds improvement. But he thought the defensive backs showed improvement, except for a couple of blown assignments on long touchdowns. Cornerback Jason Verrett, Patterson said, has "really come on the last three games" since getting beat several times at Baylor."Defensively, we played better than we did the last two games," he said. "We played harder and we played better. We just didn't make some plays we needed to make. And we made some mistakes we didn't need to make. Right now, we're deficient in some places we weren't a year ago, so we have to get better by growing up."Questionable callsGary Patterson declined to comment on several controversial penalties called against TCU. Jason Verrett intercepted an SMU pass on the Mustangs' first drive, but it was wiped away when he was called for interference. The official on the field called Sam Carter's No. 17 for the infraction, but Carter wasn't on the field. The Mustangs moved up to the Frogs' 15-yard line with the penalty and took a 7-0 lead three plays later.Also, Stansly Maponga was called for two penalties that drew second looks, including a hand-to-the-face penalty in the second quarter, and a roughing-the-passer call early in the fourth quarter. The 15 yards gave SMU the ball on the TCU 22, and the Mustangs took a 33-17 lead with a touchdown two plays later.Patterson declined to comment after the game Saturday but did point out that it was a Conference USA crew.Moving upDE Stansly Maponga: The sophomore forced his fourth fumble of the season, had two sacks for combined losses of 13 yards and finished with seven tackles against SMU. Maponga trails only defensive back Tekerrein Cuba for the team lead in tackles with 26. Maponga leads the Frogs with 4 1/2 sacks for a combined loss of 33 yards.Moving downFans leaving early: It appeared that a couple of thousand fans left Amon G. Carter Stadium early after SMU scored a touchdown on the second-half kickoff. That doesn't usually happen at big-time football schools across the nation. The majority that remained were as loud as they've ever been as TCU mounted its comeback in the fourth quarter to force overtime. But a lot of fans missed the comeback.Up next: TCU at San Diego State, 9:30 p.m. Saturday, CBS Sports Net.