That was one of my father's favorite adages, second only to "talk is cheap." It means that if you eat your cake, you don't have it anymore. You must choose one or the other. You can't have both.
The cake adage came to mind when I was thinking about Hilary Putnam (1926-2016), the eminent philosopher of science who died earlier this month. In his childhood, he had some cake: Judaism. As he grew up, he traded it for a full-fledged scientific view of the world. And then he realized that with or without science, the world wasn't nearly as good without cake.
But here's the good news: In some situations you can, after all, have your cake and eat it too. You can be both a scientific rationalist and a faithful Jew.
Religion and Rationality
As an adult, Putnam reconnected with the Jewish faith he had left behind in his youth. His son wanted to have a bar mitzvah. In helping his son prepare, Putnam -- to his surprise -- found great personal meaning in his erstwhile faith. He needed somehow to make peace between his secular understanding of the universe and the truth he found in Judaism:
"As a practicing Jew, I am someone for whom the religious dimension of life has become increasingly important ... Those who know my writings from that period may wonder how I reconciled my religious streak … and my general scientific materialist worldview at that time. The answer is that I didn't reconcile them. I was a thoroughgoing atheist, and I was a believer. I simply kept these two parts of myself separate." 
Putnam found truth and value in both frames of reference, but he felt that he could not integrate them into a single logical system so he kept them separate. Ironically, he had all the pieces that he needed to solve the puzzle. I think that he solved it without realizing he'd done so.
Twenty-seven years earlier, he had written about what it meant for beliefs to be rational:
"The only criterion for what is a fact is what it is rational to accept. I mean this quite literally and across the board; thus if it can be rational to accept that a picture is beautiful, then it can be a fact that the picture is beautiful ... A statement can be rationally acceptable at a time but not true ..." 
Putnam had a broad concept of rationality.  Instead of always requiring logical proof or empirical evidence, he thought that beliefs were rational if they were reasonable in the relevant situations. Sometimes, that did require logical proof or empirical evidence -- but not always.
Rationality Depends on Purpose
What's reasonable depends on the context and on our purpose. If you want to bake a cake, you need beliefs based on logic and empirical evidence: "The procedure is: Preheat the oven to 350°F. Spray a pan with nonstick spray. Dust with cocoa powder …"
But what if you want to enjoy a cake? Enjoyment cannot be summarized in a recipe. You don't need logic or empirical evidence. You just need to have the experience and not worry about it.
Enjoying God's presence is somewhat similar. It's not about logic or empirical facts. On this point, Putnam seems to have agreed with Ludwig Wittgenstein and other philosophers:
"Wittgenstein, like Kierkegaard, would have regarded the idea of 'proving' the truth of the Jewish or the Christian or the Muslim religion by 'historical evidence' as a profound confusion of realms, a confusion of the inner transformation in one's life that he saw as the true function of religion, with the goals and activities of scientific explanation and prediction." 
The purpose of religious faith is different from that of scientific investigation, and even from that of simple things like finding our way to the grocery store. The latter try to achieve specific goals in the physical world, but the goal of religious faith is moral and spiritual. It's not a different way of doing things in the world: It's a different way of seeing, feeling, and living in the world. The question of its scientific truth is irrelevant, just as the question of moral and spiritual value is irrelevant in science.
Because the goal of religious faith is to improve us morally and spiritually, it is reasonable to have religious beliefs that help do it. Judaism does not tell us how to get to the grocery store, but while we are there, it tells us to treat other people honestly, kindly, and with respect.
Science Isn't the Only TruthNotice something else that Putnam said: "A statement can be rationally acceptable at a time but not true."
There, he uses "true" to mean "true according to science," but the implication remains the same: It's rational to believe things in a religious context that you wouldn't believe in a scientific context. That's because their purpose and the basic worldview are different. And scientific truth isn't the only kind of truth. If it were, then truth itself would not exist because it's not a scientific concept:
"A self-refuting supposition is one whose truth implies its own falsity. For example, consider the thesis that all general statements are false. This is a general statement. So if it is true, then it must be false. Hence, it is false." 
If scientific truth is the only kind of truth, then scientific truth is impossible.  Hence, there are other kinds of truth. Some of those truths are the Jewish tradition and our way of life.
Cake. It's good for you. Enjoy it.
Putnam, H. (1981), Reason, Truth, and History. Cambridge: Cambridge University Press. Kindle edition. Putnam, H. (2008), Jewish Philosophy as a Guide to Life. Bloomington: Indiana University Press.Footnotes
 Putnam, H. (2008), p. 3.
 Putnam, H. (1981), loc. 54.
 Putnam lived to be almost 90 years old and changed his mind almost as often as Bertrand Russell, who lived to be 98. However, reasonableness was a recurring theme for Putnam.
 Putnam, H. (2008), pp. 13-14.
 Putnam, H. (1981), loc. 218.  This is my argument, not Putnam's. He might have disagreed.