Do I replace coils and plugs or just valve cover gasket and hope for the best?

I would replace the valve cover gasket and spark plug tube seals first. Note which coil came from which cylinder and which coils had oil on the plugs which could potentially have damaged the coils. After replacing the valve cover gasket and related seals then see which cylinders are misfiring and replace coils as need be. If you are replacing coils inspect/replace spark plugs and save a duplication of labor later.

Probably not because you already have symptoms. Replace gasket and plugs and see if that works.