The problem with knowing when to quit is that we are often bad at predicting the future. We have no idea if something will be fruitful or work out the way we want it to.
But ultimately, it's not "success" unless you're satisfied with how your life is. Quality of life (happiness, freedom, control) is more important than any measure you put in (wealth, weight, house size, etc.).
In that sense, if you're wondering when to quit, then ask yourself: If I keep going down this path, will I be satisfied? Will I be excited? Free? In control of my life?
You ultimately won't know if people will still be buying your product 1 or 2 years from now, but you can probably estimate if running your business will leave you more satisfied and excited, as opposed to stressed, overwhelmed, miserable.
If your quality of life has to suffer in order to be successful, then I say quit it. You always have other options.