If you wanna give anything Linux/Unix a try (let's say Fedora (not hugely recommended for servers as Fedora is usually a fast paced OS, in that case you can use CentOS, from the same Fedora family but with a more stable release cycle), Ubuntu, OpenSUSE, FreeBSD or any other), you'd need to investigate a bit and get familiar with the quirks of the OS, which tend to change from distro to distro, my first advice would be to test them on virtual machines so you get to know them, usually Linux/Unix OS in their server flavor come without a GUI, as it's usually not needed, since you can always work from the terminal but the configuration process can be somewhat complex, so a good tutorial or guide can be pretty useful (i.e Fedora 23)

Other option is to use a "pre-built" OS made for NAS storage, personally I like FreeNAS, as it has a lot of different configurations that allow you to have full access to every aspect of the files server, as well as a built in redundancy system (if you configure it, a good overview of that system is made here and here the last one being a pretty overkill system lol), this option is a bit more user friendly