There was no word on what would happen to the rest of his Asian tour. His next show is set for Seoul on May 28.
McCartney, 71, had already canceled two Tokyo performances but was set to play at the fabled Budokan on Wednesday and in the western city of Osaka on Saturday. He last toured Japan in November for a series of packed concerts.
"I was really looking forward to playing in Japan again after we had such an amazing time here in November," he said in a statement, adding that it was "hugely disappointing."
"I'd like to thank my Japanese fans for their love, messages of support and understanding."
McCartney fell ill after arriving in Japan last week. Organizers said on Tuesday that doctors have ordered him to take "complete rest."