Because asking someone what religion they are assumes they are religious. For most people "I'm atheist" also means they're irreligious. If I didn't know any better I'd think you didn't use your brain at all when you made that reply.