A language where everything is an object is impossible. An object is something that groups other things together. So what are these things? Well, they're other objects. Ok, so these objects must contain some kind of data. Ok, these also contain objects. Where does it end? It ends with primitive data types. Or some kind of built-in type. It doesn't matter if they're abstract such as variants or templates, or down to the hardware such as 32 bit signed integers. These cannot be broken down. They're the basic data types that everything else is built on top.
Hmm - I wonder if this guy has heard of bytes. Or bits. Or heck, gates. The fact is, there's nothing special about 32 bit integers - they are an abstraction that is built on top of the really primitive lower bits of the system. So should we all go back to plug boards? That's where this guy's "logic" takes him.
All objects are, are a way of raising the abstraction level up another level - instead of dealing with (slightly lower level) abstractions like integers, we deal with objects. The idea is that such abstraction makes it easier to solve problems.
In a "pure object" language like Smalltalk, the complexity is carted off to the VM. In the hybrid languages, like Java and C#, the designers felt the need to share the complexity with everyone - probably because they never fully escaped from the view that the fundamental data types in C are the bottom.