Actually no - if you consider the entire world, more people are reading books than ever before due to rising literacy rates in developing nations. Besides, books have NEVER been that popular throughout history. Throughout time there has always been a core of dedicated literature fans and people who rarely read, and the people who rarely read have always made up the majority of the population.
I remember when I was in High School - few people visited the Library and most of my classmates only read what the teachers forced them to read. This was during the early 2000's and late 90's. Kids have never been big fans of books, and even most adults rarely read when they don't have to. Even BEFORE television, most people spent their time either talking, listening to the radio or drinking. There has never been an instance in history in which the majority of any population were dedicated book readers - as I just said, it's only relatively recently that more than 50% of any population could read anything more complicated than a road sign.
All in all, I wouldn't worry too much about it.