Your account, I believe, corresponds to the essence of George Berkeley’s work on perception. His polemic was with people who claimed that there was an insensible extension or Philisophic Matter, which he claimed that "Irishmen cannot attain to”. When one understands correctly what he was arguing against, the fanciful interpretation of his philosophy, which is now generally accepted, disappears.
What we call mathematics, as you say: “one” doesn’t exist since it is a unit, a conception in the mind, for Plato, was the simplest form of a manner of reliable foreknowledge.
Mathematikos originally had a larger compass. The sign on the Platonic Academy demanded that one could at least grasp arithmetic and geometry, as a stepping stone to the harder stuff,
mathematikos meant also the ideas that are the same kind of beings but more difficult to grasp. The ideas are patterns: a tree, a human being, justice. A triangle, or an integer as a unit, is also such a pattern. Plato is interested in the patterns not only as the ground of an unchanging reality, archetypes or prototypes that always are, but also, one might say, as something for open contemplation. Insofar as one insist that Plato is identical to this or that discussion that occurs in the dialogues of Plato, e.g., to understand the ideas in the manner of a Penrose, one doesn't enter into the Greek thinking. Yet, that is the most common way to treat Plato.
“So this way you can manipulate and control the average idiot much better.”
I wouldn’t underestimate the lack of ability to think of most persons, even those in the highest places. Usually these things are not sinister. The account in Locke of primary and secondary substances is very powerful. It leads to the “two tables” account. Scientific and ordinary reality. Though, that said, one never knows what machinations are at work to vicious purpose.
All this leads to a radical mystery. Since one is left to the unassisted human intelligence to interpret reality. One has a correct memorey of how to make things function which calls out, as it were, for interpretation.
“ since a better understanding of the ontological reality might lead to more functional mathematical models, there are still theoretical physicists.”
The mighty Feynman had this to say:
https://www.youtube.com/watch?v=NM-zWTU7X-k