Your variable table can get bloated under three circumstances:

1. Your site has existed since 4.7 or earlier, back when modules didn't have a way to clean up their variables.
2. You've tried out many modules, but either forgot to uninstall the module before removing the code, or the module didn't remove its variables during uninstall.
3. A poorly coded module has been installed that improperly stores a lot of data in the variables table.

This module can help you cleanup from the first two problems. For #3 you should file an issue with the module in question.

If your variable table is bloated, you may see random memory limit fatal errors (especially on shared hosting where PHP memory limits are sometimes very low), or just generally non-optimal performance.

Warning: Use this module at your own risk. While the module does gracefully handle dynamic variables, it makes no guarantees not to break your site. Smart developers run backups before doing something that alters the database.