The minimum 1GB is stated because the installation process which uses yarn and npm can use that much memory so you need at least that or some swap space to ensure the normal installation procedure works. If you are using a different installation process such as pre-built containers then the memory requirements may be different.
When running Ghost typically uses ~250MB of memory for small-moderate sized blogs but it might spike higher during certain operations. It’s always recommended to monitor memory and performance for your particular content and traffic patterns so that you can plan your infrastructure appropriately.
does it affect performance if we don’t use swap memory?
As stated above swap will likely only be used during installation so shouldn’t have any effect on the performance of your blog. Swap space is slow compared to memory so it’s always preferable for it not to be used by running programs. However simply having swap space available on a server will not affect performance one way or the other - you need to monitor your memory usage to see if swap is being used due to high memory usage, if it is then increase the amount of memory you have.
If you’re unsure what swap memory is I recommend reading up on it - What is swap memory? - Server Fault
does this refer to default theme or custom made theme
It’s referring to custom themes. When you create a theme you have all of the theme API available to you so it’s possible to use it in ways that cause performance problems similar to how you can create performance issues with general programming languages. The default theme is not known to have any performance issues.