“Let someone else take over his office” (Acts 1:20). It says later in the chapter that St. Matthias (who was chosen to take Judas Iscariot’s office) was listed among the twelve apostles, and some suppose that merely to restore the original number of twelve was the sole purpose of his election. But it’s clear that doing so was not in any way simply numerically desirable because the Lord appointed St. Paul to be an apostle as well. Consequently twelve was not a special thing in itself, but rather what was necessary was that the office be filled and not merely that a number be satisfied (else a thirteenth apostle would break the numbering). Furthermore, it would be superfluous to replace Judas if the expectation was that apostolic authority was to end with the death of the apostles, because they were all going to die anyway. Therefore it was not merely a number but rather an office that had to be sustained (as the passage says).
How then are they to call on him if they have not come to believe in him? And how can they believe in him if they have never heard of him? And how will they hear of him unless there is a preacher for them? And how will there be preachers if they are not sent? [Romans 10:14-15]
Preachers must be sent. That is, they must be authorized. But it is impossible to identify an authorized preacher apart from a visible Church. Likewise the visible Church must be that which Christ founded. Consequently its authority must be visibly preserved from generation to generation, and this can only achieved by means of apostolic succession.
[Update, 12/22/2012: also see here.]