O'Connor's header five minutes from time came against the run of play in the second half, which will have been a relief for top scorer Billy Sharp, who missed a host of chances for the Championship side.
After two postponements and 17 days since the original fixture was due to be played, Griffin Park was finally deemed fit to stage this tie.
The delay had no effect on Doncaster as the Championship side got off to a fast start, creating two chances in the opening few minutes.
First Dean Shiels beat Leon Legge on the left and whipped a ball into the middle and Sharp pulled his shot just wide of Lewis Price's goal.
And moments later the striker was through again but dragged his shot agonisingly wide of the far post.
Brentford got into the game and could have taken the lead after 15 minutes when Myles Weston's perfectly flighted ball was met by Charlie MacDonald at the far post but he headed straight at Neil Sullivan.
And midway through the first half the Bees went close again when Ben Strevens' glancing header went just wide.
But just before the break, Doncaster showed their threat again as Sharp went through and rounded keeper Price, who was left helpless on the floor, but the striker could not add to his 11 goals this season as he missed the target with the goal open.
Price made up for his mistake moments later as he produced a brilliant save to thwart Sharp again when striker shot from the edge of the area.
Brentford proved to be the most likely to take the lead in the second half with Weston causing havoc down the Doncaster right-hand side but he lacked the killer ball when it mattered.
But with just five minutes remaining, O'Connor rose to head home the winner from a Brian Stock cross to ensure it is Doncaster who travel to Pride Park on Saturday.