One virtual host or multiple in Apache, Nginx, and other webservers?

Does putting all your configurations into a single file in Apache, Nginx, and other similar webservers make a given webserver slower or faster as compared to using separate virtual hosts with separate configuration files for each site?

Basically is putting everything into

/etc/apache-or-nginx/sites-available/yoursite 
faster than creating
/etc/apache-or-nginx/sites-available/yoursite1
/etc/apache-or-nginx/sites-available/yoursite2
/etc/apache-or-nginx/sites-available/yoursite3

- - , .
.