Pythagoras believed in a highly idealized world, as did all the major Greek thinkers; most were extremely disinterested in observation. Hence, Aristotle's (now seemingly ridiculous) views on motion dominated for 2000 years. In modern physics, numbers are used to make approximations of the world. A numerical solution to a problem is only as good as the model one is using, so to speak, and there is no absolutely perfect model.

As to irreducible levels of order, Wittgenstein put it like this: We could try to understand what is happening in a room by describing all that is going on that can be observed. We could observe a man sitting in an elevated wooden bench, and several other men sitting at table opposite him. There also exist 12 individuals paying attention to each person as they speak in succession and another near the front transcribing everything, and etc., etc., etc. We could describe the entire scene in exquisite detail, but unless we already knew what jury trial is, it would be a complete waste of time. There is no amount of description that can allow us to understand what is happening.

Kauffman makes the same argument about biology. We spend so much time with a reductionist approach, but perhaps we can never in principle discern how these findings relate to the organismal level, which results from essentially infinite interactions smaller units on many different length and time scales.

I would totally agree with you if you said "Numbers are the only form of absolute Truth". I just don't think they can always provide a useful description of the world.