During residency, I worked hospital shifts that could last 36 hours, without sleep, often without breaks of more than a few minutes. Even writing this now, it sounds to me like I’m bragging or laying claim to some fortitude of character. I can’t think of another type of self-injury that might be similarly lauded, except maybe binge drinking. Technically the shifts were 30 hours, the mandatory limit imposed by the Accreditation Council for Graduate Medical Education, but we stayed longer because people kept getting sick. Being a doctor is supposed to be about putting other people’s needs before your own. Our job was to power through.

The shifts usually felt shorter than they were, because they were so hectic. There was always a new patient in the emergency room who needed to be admitted, or a staff member on the eighth floor (which was full of late-stage terminally ill people) who needed me to fill out a death certificate. Sleep deprivation manifested as bouts of anger and despair mixed in with some euphoria, along with other sensations I’ve not had before or since. I remember once sitting with the family of a patient in critical condition, discussing an advance directive—the terms defining what the patient would want done were his heart to stop, which seemed likely to happen at any minute. Would he want to have chest compressions, electrical shocks, a breathing tube? In the middle of this, I had to look straight down at the chart in my lap, because I was laughing. This was the least funny scenario possible. I was experiencing a physical reaction unrelated to anything I knew to be happening in my mind. There is a type of seizure, called a gelastic seizure, during which the seizing person appears to be laughing—but I don’t think that was it. I think it was plain old delirium. It was mortifying, though no one seemed to notice.

No matter what happened to my body, I never felt like it was dangerous for me to keep working. I knew I was irritable and sometimes terse, and I didn’t smell the best, but I didn’t think anything I did was unsafe. Sleep experts often liken sleep-deprived people to drunk drivers: They don’t get behind the wheel thinking they’re probably going to kill someone. But as with drunkenness, one of the first things we lose in sleep deprivation is self-awareness.

It’s this way of thinking—that you can power through, that sleep is the easiest corner to cut—that makes sleep disturbance among the most common sources of health problems in many countries. Insufficient sleep causes many chronic and acute medical conditions that have an enormous impact on quality of life, not to mention the economy. While no one knows why we sleep, it is a universal biological imperative; no animal with a brain can survive without it. Dolphins are said to sleep with only half their brain at a time, keeping partially alert for predators. Many of us spend much of our lives in a similar state.

Since my residency, I’ve become sort of obsessive about sleep—how much we really need, how to optimize it, whether there are ways to game the system. What can be said definitively about sleep and wakefulness? What I’ve found is a perpetual divide between what’s known to scientists and what most people do.

How much sleep do I actually need?

One 2014 study of more than 3,000 people in Finland found that the amount of sleep that correlated with the fewest sick days was 7.63 hours a night for women and 7.76 hours for men. So either that is the amount of sleep that keeps people well, or that’s the amount that makes them least likely to lie about being sick when they want to skip work. Or maybe people who were already sick with some chronic condition were sleeping more than that—or less—as a result of their illness. Statistics are tough to interpret. Isolated studies are tougher. That’s why the American Academy of Sleep Medicine and the Sleep Research Society convened a body of scientists from around the world to answer this question through a review of known research. They looked at the effects of sleep on cardiovascular disease, cancer, obesity, cognitive failure, and human performance, vetting each paper based on its scientific strength.

The consensus: Most adults function best after seven to nine hours of sleep a night. Going to sleep and waking up at consistent times each day is valuable too. When we get fewer than seven hours, we’re impaired (to degrees that vary from person to person). When sleep persistently falls below six hours per 24, we are at an increased risk of health problems.

Can I train myself to need less sleep?

As an experiment for his high-school science fair in 1964, a 17-year-old San Diego boy named Randy Gardner stayed awake for 264 hours. That is 11 days. Since 1964, the standards for science-fair safety have changed.

The project attracted the attention of the Stanford sleep researcher William Dement, among others. Dement and other researchers took turns watching and assessing the young man’s consciousness. By all accounts, he took no stimulant medications. Nor did he seem to suffer any permanent deficits. Dement said that on day 10, Gardner even beat him at pinball. The boy later said of his experiment that the key to staying awake was “just talking yourself into it.”

I asked David Dinges, the chief of the division of sleep and chronobiology at the University of Pennsylvania, how many people could do anything close to that without dying. He replied that “when animals are sleep-deprived constantly, they will suffer serious biological consequences. Death is one of those consequences.”

That said, cases like Gardner’s—of people who suffered great sleep deprivation without major setbacks—are well documented. A small number of people, sometimes called “short sleepers” and commonly thought to make up perhaps 1 percent of the population, seem to thrive on only four or five hours a night. Dinges said that “we probably do have people among us—and not necessarily 1 percent; there may be many more than that—who can actually tolerate sleep loss better than others.” This proposition has been borne out in studies of participants in transoceanic sailing races, which did not afford them the luxury of long blocks of sleep. The winners tended to be the people who slept the least, often in multiple short bursts.

The concept of sleeping in short bursts has spread since those races began, in the 1960s. Today, a small global community of people practices “polyphasic sleeping,” based on the idea that by partitioning your sleep into segments, you can get away with less of it.

Though it is possible to train oneself to sleep in spurts instead of a single nightly block, Dinges says it does not seem possible to train oneself to need less sleep per 24-hour cycle. And he notes that even for the 1 percent (or so) who can survive on less sleep and function well cognitively, we still don’t know how the practice might be affecting metabolism, mood, and myriad other factors. “You may be cheerful, but not cognitively fit. Or you may be cognitively fit, but hard to be around because you’re pushy or hyperactive.”

Around the time of Gardner’s historic science project, the U.S. military got interested in sleep-deprivation research: Could soldiers be trained to function in sustained warfare with very little sleep? The original studies seemed to say yes. But when the military put soldiers in a lab to make certain they stayed awake, performance suffered. Cumulative deficits accrued with each night of suboptimal sleep. The less sleep the soldiers got, the more deficits they suffered the next day. But as with my own residency experience, they couldn’t tell that they had a deficit.

“They would insist that they were fine,” said Dinges, “but weren’t performing well at all, and the discrepancy was extreme.”

This finding has been replicated many times over the intervening decades, even as many professions continue to encourage and applaud sleep deprivation. In one study published in the journal Sleep, researchers kept people just slightly sleep deprived—allowing them only six hours to sleep each night—and watched the subjects’ performance on cognitive tests plummet. The crucial finding was that throughout their time in the study, the sixers thought they were functioning perfectly well.

Effective sleep habits, like many things, seem to come back to self-awareness.

I drink coffee instead of sleeping, so I’m fine.

Caffeine is the most consumed stimulant in the world. The chemical induces reactions throughout the body that normally occur in intense situations. When we sense danger, for example, the pituitary gland activates the adrenal glands to secrete epinephrine, or adrenaline, into our blood. Adrenaline is the hormone that’s meant to be released when we are under stress and need to muster energy to, say, outrun a bear or lift a fallen boulder off our climbing partner. (He’s probably not alive anymore, but it’s worth checking.) Caffeine increases adrenaline levels in the blood. It has repeatedly been shown to improve athletic performance in the short term, from how high a person can jump to how fast a person can swim.

The hormone surge also creates a buzz. To lift that boulder we need a flood of energy to fuel our muscles, but first we need to think we can lift the boulder. The “psychoactive” component of caffeine is what makes anything seem possible when brainstorming during your third hour in a coffee shop.

Caffeine works primarily by blocking the action of a chemical called adenosine, which slows down our neural activity, allowing us to relax, rest, and sleep. By interfering with it, caffeine cuts the brake lines of the brain’s alertness system. Eventually, if we don’t allow our body to relax, the buzz turns to anxiety.

Thanks to caffeine, many of us stimulate that fight-or-flight response not just occasionally, under dire circumstances, but daily, in our offices. Eighty-five percent of U.S. adults consume some form of caffeine most days, with an average daily dose of 300 milligrams (roughly 27 ounces of coffee). Strategic use of small amounts of caffeine can be cognitively advantageous, but at such a high dose, caffeine is likely to throw off our sleep and energy cycles in the long term, altering the body’s internal clock. At that point, many people go in search of products to help them sleep.

But there’s no real danger in consuming a lot of caffeine, right? Can’t caffeine make you live longer?

We frequently hear that drinking a small amount of coffee can be healthy. This always comes back to the evidence that some coffee-drinking is a common behavior among long-lived, healthy people across populations. News stories tend to interpret this evidence optimistically, reporting that coffee may be good for you. In reality, it might just be an interesting correlation. Randomized, controlled trials on nutrition are extremely difficult to conduct, as the effects of dietary changes are complex and often take years, if not a lifetime, to reveal themselves.

Those who claim that coffee is healthful tend to point to its high level of antioxidants. But antioxidant supplements have not been proved to correlate with health or longevity. Antioxidants represent a vast spectrum of substances. Vitamin E is an antioxidant, and taking vitamin-E supplements has been shown to increase men’s risk of prostate cancer.

If coffee does have an effect on longevity, it is likely a result of something more global than the potential effect of antioxidants—such as the fact that constant exposure to caffeine, even at low levels, suppresses appetite (in a world where most people eat more than is ideal). Or that it encourages social interaction—it inclines us to go out and do things with people—which itself is generally beneficial to health. These are legitimately positive results. But as with all chemicals, the comprehensive effect of caffeine on our health depends on how, and how much, we use it.

In 2013, a 24-year-old advertising copywriter in Indonesia died after prolonged sleep deprivation, collapsing a few hours after tweeting “30 hours of working and still going strooong.” She went into a coma and died the next morning. A family acquaintance wrote on Facebook, “She died because too much of overtime working, and too much kratingdaeng attacks her heart.” Kratingdaeng is the Thai name for the product known elsewhere as Red Bull.

Sleep deprivation is clearly linked to heart disease and strokes. Beyond that, the vitamin/caffeine/amino-acid concoctions known collectively as energy drinks have been implicated in thousands of emergency-room visits in recent years; energy-drink-related ER visits doubled from 2007 to 2011, according to the U.S. Substance Abuse and Mental Health Services Administration. For now, this is simply a correlation, with a plausible explanation that one could be causing the other; it is not proof of harm. And yet, notes Michael Jacobson, the head of the Center for Science in the Public Interest, “there are several fatalities possibly related to energy drinks, and several lawsuits. In some people, it appears to be due to underlying heart defects—when they get this dose of caffeine, they succumb.”

Although the FDA warns us rather unambiguously that “caffeine overdose is dangerous and can kill you,” I’ve not seen that happen, and Jacobson, a public-health advocate, confirms that except at extraordinarily high levels, caffeine isn’t known to kill otherwise healthy people. It may not be the sole culprit in hospitalizations related to energy drinks. After all, many of the people who have been hospitalized after consuming energy drinks are presumably also coffee drinkers, notes Jacobson—but few, if any, have been made acutely ill by coffee.

I can’t sleep. Is my phone really keeping me up? Should I seriously not be reading my phone in bed? That seems impossible.

The United Nations declared 2015 to be the International Year of Light and Light-Based Technologies. That summer, the New York Blue Light Symposium brought together experts who are trying to reckon with the invasion of all this new light into our lives. A keynote speaker was the Japanese ophthalmologist Kazuo Tsubota, the president of the International Blue Light Society, which aims to “promote public awareness of pertinent research on the physical effects of light.” Its founding followed a 2012 report by the American Medical Association titled “Light Pollution: Adverse Health Effects of Nighttime Lighting.”

Of all the things to have health concerns about, nighttime lighting? Well, yes. When light enters your eye, it hits your retina, which relays signals directly to the core of your brain, the hypothalamus. The size of an almond, the hypothalamus has more importance per volume than any other part of your body. Yes, that includes the sex organs—you would have no sex drive without the hypothalamus. This almond is the interface between the electricity of the nervous system and the hormones of the endocrine system. It takes sensory information and directs the body’s responses, so that the body can stay alive.

Among other roles in maintaining bodily homeostasis—appetite, thirst, heart rate, etc.—the hypothalamus controls sleep cycles. It doesn’t bother consulting with the cerebral cortex, so you are not conscious of this. But when your retinas start taking in less light, your hypothalamus assumes it’s time to sleep. So it wakes up its neighbor the pineal gland and says, “Hey, make some melatonin and shoot it into the blood.” And the pineal gland says, “Yes, okay,” and it makes the hormone melatonin and shoots it into the blood, and you become sleepy. In the morning, the hypothalamus senses light and tells the pineal gland to stop its work, which it does. Test your blood for melatonin during the daytime, and you will find almost none.

All of this is why we’re told to minimize screen time before bed. Phones and tablets emit light that’s skewed heavily toward the blue end of the visible spectrum, and some research suggests that these frequencies are especially influential in human sleep cycles. Using a “night mode,” available on some phones, is supposed to minimize that effect. That’s probably worth doing—so long as you don’t end up canceling out any benefit by spending more time looking at the lit screen.

Can’t I just take a melatonin supplement if I can’t get to sleep?

Melatonin is one of the very few hormones that you can purchase in the United States without a prescription. It is considered a dietary supplement and therefore held to essentially no premarket standards of quality, safety, or efficacy. The pharmacist can’t give me the eye drops that help control my glaucoma without a prescription. The pharmacist can’t give insulin to a diabetic person without the recurring order of a doctor, to which not all people have easy access. But melatonin, which tinkers with the work of the most crucial part of your brain? It’s over there in Aisle 5. Buy as much as you like. It’s next to the caffeine pills.

In 2015, Ben Yu, who’d dropped out of Harvard to form a biotech start-up, launched a product called Sprayable Sleep, which contains melatonin. Spray it onto your skin, and it’s supposed to put you to sleep. (Sprayable Sleep is the company’s second product. Its first was the perfect complement: Sprayable Energy, or topical caffeine.)

When I spoke with Yu, he referred to melatonin not as a hormone but as a “biological signaling molecule.” I asked him whether he did this because customers might be averse to spraying themselves with a hormone. “I thought that might be a loaded word,” he agreed, “but it turns out, people don’t seem to care.”

In a sleep-deprived culture, the promise of sleep can lead people to abandon caution. In its initial crowd-funding campaign on Indiegogo, Sprayable Sleep raised 9,798. (That’s 2,106 percent of what the company set out to raise, collected from nearly 5,000 people.)

Unlike melatonin pills, which are absorbed into and eliminated from the blood before the night is over, Sprayable Sleep is supposed to keep you asleep through the night, as the hormone gradually percolates through your skin and into your bloodstream. I tried it for a couple of weeks, and I did sleep, but it was tough to tell what effect the product was having: I sleep most nights. That said, I can confirm that it didn’t burn my skin. Also, that people don’t like it when you pretend you are going to spray it on them.

Melatonin supplements have been shown to make some people fall asleep more quickly, but they aren’t proven to increase the total time or quality of sleep. And of course, as with most things sold as supplements in the United States, the effects of long-term use are unknown.

What is clear is that supplement overuse can be dangerous. Melatonin is crucial to the functioning of the most finely tuned apparatuses in the body, and David Dinges is especially concerned about its use by young people. As he put it, “No child should have a melatonin supplement—or a caffeinated drink—without a doctor being involved.” Adults, he says, “at least might make informed decisions.”

The delicate word there is informed. Many people seem engaged in a daily arms race between wakefulness and unconsciousness, using various products to mask and manage poor sleep habits, and ultimately just needing more products. Spray-on caffeine followed by spray-on melatonin. Or alcohol, which only further messes with our physiological rhythms.

So how does one break that cycle? Factors outside of our control—jobs and families and light pollution, to name a few—can make this hard to do. But when possible, here are a few simple ideas that many experts recommend. Try to keep a somewhat constant bedtime and wake-up time, even on weekends. Keep caffeine use moderate, even if you don’t feel like a nighttime coffee affects you. The same goes for nightcaps. (Not necessarily a joyless suggestion—maybe you can meet a friend for a beer at 4 p.m. instead of 10 p.m.) Use screens judiciously, too. Remember that even on night mode, a phone is shooting light into your brain. Have sex with someone instead. Or, sometimes preferable, read something on paper.