Yes. You should dip the rag and completely seal the bottle before lighting and throwing. An open container of gas around flames is dangerous.
The original molotov cocktails actually just used storm matches on the sides- The point is to spread gas and light it, not make an improvised explosive.