Danvers State Hospital
I had heard of medical designs that employ individuals who supply Bayesian subjective priors that are deemed either “enthusiastic” or “skeptical” as regards the probable value of medical treatments.[i] From what I gather, these priors are combined with data from trials in order to help decide whether to stop trials early or continue. But I’d never heard of these Bayesian designs in relation to decisions about building security or renovations!
You may have heard that the Department of Homeland Security (DHS), whose 240,000 employees are scattered among 50 office locations around D.C.,has been planning to have headquarters built at an abandoned insane asylum St Elizabeths in DC [ii]. See a recent discussion here. When they first submitted its plan in 2006 officials projected the new facility would be ready by 2015. Now an additional $3.2 billion is needed to complete the renovation of the 159-year-old mental hospital by 2026 (Congressional Research Service)[iii].The initial plan of developing the entire structure is no longer feasible; so to determine which parts of the facility are most likely to be promising, “DHS is bringing in a team of data analysts who are possessed” said Homeland Security Secretary Jeh Johnson (during a DHS meeting, Feb 26) –“possessed with vibrant background beliefs to sense which buildings are most probably worth renovating, from the point of view of security. St. Elizabeths needs to be fortified with 21st-century technologies for cybersecurity and antiterrorism missions”.
Failing to entice private companies to renovate the dilapidated west campus of the historic mental health facility that sits on 176 acres overlooking the Anacostia River,they can only hope to renovate selectively: “Which parts are we going to overhaul? Parts of the hospital have been rotting for years!”Johnson declared.
Read more:
[I’m too rushed at the moment to write this up clearly but thought it of sufficient interest to post a draft (look for follow-up drafts).]
Excerpt from DHS memo:
The description of the use of so-called “enthusiastic” and “skeptical” priors is sketched in a DHS memo released in January 2014 (but which had been first issued in 2011). Here’s part of it:
Enthusiastic priors are used in evaluating the portions of St. Elizabeths campus thought to be unpromising, in terms of environmental soundness, or because of an existing suspicion of probable security leaks. If the location fails to be probably promising using an enthusiastic prior, plus data, then there is overwhelming evidence to support the decision that the particular area is not promising.
Skeptical priors are used in situations where the particular asylum wing, floor, or campus quadrant is believed to be probably promising for DHS. If the skeptical opinion, combined with the data on the area in question, yields a high posterior belief that it is a promising area to renovate, this would be taken as extremely convincing evidence to support the decision that the wing, floor or building is probably promising.
But far before they can apply this protocol, they must hire specialists to provide the enthusiastic or skeptical priors. (See stress testing below.) The article further explains, “In addition, Homeland Security took on a green initiative — deciding to outfit the campus’ buildings (some dating back to 1855) with features like rainwater toilets and Brazilian hardwood in the name of sustainability.” With that in mind, they also try to get a balance of environmentalist enthusiasts and green skeptics.
Currently “I think that the morale of DHS, unity of mission, would go a long way if we could get to a headquarters,” Mr. Johnson said. He was pleased to announce that an innovating program of recruiting was now complete.
Stress Testing for Calibrated “Enthusastic” and “Skeptical” Prior Probabilities
Perhaps the most interesting part of all this is how they conduct stress testing for individuals to supply calibrated Bayesian priors concerning St. Elizabeths. Before being hired to to give “skeptical” or “enthusiastic” prior distributions, candidates must pass a rather stringent panoply of stress tests based on their hunches regarding relevant facts associated with a number of other abandoned insane asylums. The list of asylums on which they based the testing (over the past years) has been kept Top Secret Classified until very recently [iv]. Even now,one is directed to a non-governmental website to find a list of 8 or so of the old mental facilities that apparently appeared in one batch of tests.
Scott Bays-Knorr, a DHS data analyst specialist who is coordinating the research and hiring of “sensors,” made it clear that the research used acceptable, empirical studies: “We’re not testing for paranormal ability or any hocus pocus. These are facts, and we are interested in finding those people whose beliefs match the facts reliably. DHS only hires highly calibrated, highly sensitive individuals”, said Bays-Knorr.
They combine written tests with fMRI data— which monitors blood flow and, therefore, activity inside the brain in real time —to try to establish a neural signature that can be correlated with security-relevant data about the abandoned state hospitals. “The probability they are attuned to these completely unrelated facts about abandoned state asylums they’ve probably never heard of is about 0. So we know our results are highly robust,” Bayes-Knorr assured some skeptical senators.
Danvers State Hospital
Take for example,Danvers State Hospital, a psychiatric asylum opened in 1878 in Danvers, Massachusetts.
“We check their general sensitivity by seeing if any alarm bells go off in relation to little known facts about unrelated buildings that would be important to a high security facility. ‘What about a series of underground tunnels’ we might ask, any alarm bells go off? Any hairs on their head stand up when we flash a picture of a mysterious fire at the site in 2007?” Bays-Knorr enthused. “If we’ve got a verified fire skeptic who, when we get him to DC, believes that a part of St. Elizabeths is clear, then we start to believe that’s a fire-safe location. You don’t want to build U.S. cybersecurity if good sensors give it a high probability of being an incendiary location.”
Some of the tests involve matching drawn pictures, which remind me a little of those remote sensing tests of telepathy. Here’s one such target picture:
Target: Danvers State Hospital
They claim they can ensure robustness by means of correlating a sensor’s impressions of completely unrelated facts about the facility. For example, using fMRI data they can check if “anything lights up” in connection with Lovecraft’s Arkham Sanatorium”—apparently a short story “The Thing on the Doorstep”, or Arkham Asylum in the Batman comic world. I frankly don’t get it, but then again, I’m highly skeptical of approaches not constrained by error statistical probing.
Yet Bayes-Knorr seemed to be convincing many of the Senators who will have to approve an extra 3 billion on the project. He explained, “We never published this list of asylums, the candidates did not even know what we were gong to ask them. It doesn’t matter if they’re asylum specialists or have a 6th sense. If they have good hunches, if they fit the average of the skeptics or the enthusiasts, then we want them.” Only if the correlations are sufficiently coherent is a ‘replication score’ achieved. The testing data are then sent to an independent facility of blind “big data” statisticians,Cherry Associates, from whom the purpose of the analysis is kept entirely hidden.Testing has gone on for the past 7 years and is only now winding up.
Community priors are eventually obtained based on those hired as U.S. Government DHS Calibrated Prior Degree Specialists.
Sounds like lunacy to me.
[i]Spiegelhalter, D. J., Abrams, K. R., & Myles, J. P. (2004). Bayesian approaches to clinical trials and health care evaluation. Chichester: Wiley.
[ii]Homepage for DHS and St. Elizabeths Campus Plans.
http://www.stelizabethsdevelopment.com/index.html
GSA Development of St. Elizabeths campus:“Preserving the Legacy, Realizing Potential”
[iii]
U.S. House of Representatives
Committee on Homeland Security
January 2014
Prepared by Majority Staff of the Committee on Homeland Security
http://homeland.house.gov/sites/homeland.house.gov/files/documents/01-10-14-StElizabeths-Report.pdf
[iv] The randomly selected hospitals in one standardized test included the following:
Topeka State Hospital
Danvers State Hopital
Denbigh Asylum
Pilgrim State Hospital
Trans-Allegheny Asylum
High Royds Hospital
Whittingham Hospital
Norwich State Hospital
———–
Filed under: junk science, subjective Bayesian elicitation
