"In my opinion I think he's tampered with the ball and used an object to do that," former South African captain Graeme Smith said.
"It does look like it's a bit of sandpaper. The footage doesn't look good.
"You can see that he's taking something out of his pocket, using it on the ball, and once the message comes out (to the umpires) he's clearly hiding it away. When he gets approached he takes out the old sunglasses cleaner (from his pocket)."
Talking on SuperSport, Shane Warne also expressed concern about what he had seen.
"You've got to own up and say what was it that you were hiding," Warne said. "You can't have that in the game.
"We've got to get to the bottom of it. You know when you get caught you've got to own up and be honest. The Aussies have to be honest and say 'this is how it happened'."
Former Australian captain Allan Border added: "It is very suspicious. There is no doubt about that. If you're caught doing the wrong thing, you've got to pay the penalty."
Dale Steyn, the injured South African fast bowler who is at Newlands, had immediately tweeted a screengrab, accompanied by the words: "Can we talk about this?"
Commentators in the broadcast box were heard saying "Oh no" as the footage was shown.
The umpires chose not to replace the ball after speaking to Bancroft.
"I'm quite amazed the umpires haven't done anything with the ball. The footage is quite damning," Smith said in commentary.
"For me it's quite obvious that he's doing something with the ball and the umpires need to do something about it."
Both Warne and Smith said they did not believe that Bancroft would have decided himself to use a foreign object on the ball.
"If it is proved that what has gone on in the footage is correct then some tough questions have to be asked of Steve Smith and Darren Lehmann," Smith said.
"I think there is a lot of questions that need to be answered and Australia need to answer them. The leadership needs to answer for that in my opinion."
The last series between Ausralia and South Africa in 2016/17 featured a ball-tampering controversy when Proteas captain Faf du Plessis was found guilty of using a mint to shine the ball.
Under the ICC code of conduct is a level-two offence, carrying up to four demerit points, a total that brings about a one-Test ban for a player found guilty.
Chris Barrett is a Sports Writer with The Sydney Morning Herald.
Morning & Afternoon Newsletter