Most governments have a national bank. For example Bank of England or the Federal Reserve. And the different parts of the governments have bank accounts so they can interact with the rest of the banks and keep track of their spending and income.
However, there can be a lot of things happening behind the scenes depending on the country. Normally the government is the one who prints money so when you pay taxes they can just burn the money you just paid and when they pay a bill they can just print the money they need. So a government bank account may not even have a balance. Income and expenses is being tracked like normal with budgets and ledgers but the actual money does not have to be tracked.
But printing money when you need it is not the best option as it causes inflation which can cause economic collapse. So the central bank will issue bonds to loan the money it needs. And when it gets income it will be used to pay back the bonds.
The exact details of how a government handles its money is up to them but as you can see they have a few more options then a normal business. They do not even have to follow the banking laws as the laws that decide how the government should handle its money can override the banking laws.