Occupancy Spike Detection

Toronto Shelter System — days unusually above seasonal and trend expectations

Retrospective spike detection for Toronto’s daily shelter occupancy — identifying days well above seasonal and trend expectations with Seasonal-Trend decomposition using LOESS (Locally Estimated Scatterplot Smoothing), abbreviated STL, and robust thresholds.

A spike is a day when the shelter system housed substantially more people than season and trend would predict — not merely a busy day. The system follows well-established patterns: occupancy rises in winter as cold weather drives demand, and falls in summer. It also carries a long-run trend that reflects structural changes in the city’s housing market and refugee policy. This analysis uses STL decomposition to remove both of those forces mathematically, leaving behind only the part that cannot be explained. A spike is a day when that unexplained remainder is unusually large. The charts on this page apply spike detection to the City’s Daily Shelter & Overnight Service Occupancy & Capacity data — an analysis that, as far as can be determined, has not been published before.

This approach measures individuals accommodated (SERVICE_USER_COUNT in the City’s open data), not the occupancy rate. The distinction matters. A count spike unambiguously means more people were in the system on that day. A rate spike could mean more people, or it could mean fewer available beds — two very different situations with different policy implications. Count is the right signal for detecting stress events.

This is a retrospective, descriptive analysis. It identifies days that were statistically anomalous given what the system had been doing in the preceding two years — it does not predict future spikes. The thresholds are set against the system’s own recent behaviour, not against any external standard of what occupancy levels should look like. A predictive spike-probability model is planned for a future update.

The page shows two threshold bands. The amber band (2 MAD-SD, or two Median Absolute Deviation standard deviations above the seasonal-trend expected value) flags more days and catches subtler anomalies — useful for building a comprehensive record. The red band (3 MAD-SD) flags only pronounced spikes unlikely to arise from routine daily noise. Both thresholds are calibrated against the most recent two years of system behaviour, so the bar rises or falls as the system becomes more or less volatile over time.

data = FileAttachment("../data/spike_detection.json").json()

// Parse dates — spike_detection.json has date as "YYYY-MM-DD" string
spikeData = data.map(d => ({
  ...d,
  dateObj: new Date(d.date + "T00:00:00")   // avoid UTC/local midnight shift
}))

windowOptions = [
  {label: "1 Month",  months: 1},
  {label: "3 Months", months: 3},
  {label: "6 Months", months: 6},
  {label: "12 Months", months: 12},
  {label: "All (2021–present)", months: null}
]
viewof selectedWindow = Inputs.select(windowOptions, {
  label: "View window",
  format: d => d.label,
  value: windowOptions[4]   // default: All
})

// Band toggle — toggleable shaded regions
viewof showBands = Inputs.checkbox(["2 MAD-SD", "3 MAD-SD"], {
  label: "Show threshold bands",
  value: ["2 MAD-SD", "3 MAD-SD"]
})

windowCutoff = selectedWindow.months === null
  ? new Date("2021-01-01")
  : (() => {
      const d = new Date()
      d.setMonth(d.getMonth() - selectedWindow.months)
      return d
    })()

// Filter system-wide series to selected window (zoom only)
systemData = spikeData
  .filter(d => d.series === "system" && d.dateObj >= windowCutoff)
  .sort((a, b) => a.dateObj - b.dateObj)

// Men gets its own full-width chart; remaining four go into the small-multiples grid
menData = spikeData
  .filter(d => d.series === "men" && d.dateObj >= windowCutoff)
  .sort((a, b) => a.dateObj - b.dateObj)

sectorOrder = ["women", "mixed_adult", "youth", "families"]
sectorLabels = ({women: "Women", mixed_adult: "Mixed Adult", youth: "Youth", families: "Families"})

MON = ["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"]

xAxis = (() => {
  if (!systemData.length) return {type: "time", label: null}
  const spanDays = (systemData[systemData.length-1].dateObj - systemData[0].dateObj) / 86400000
  const spanMonths = spanDays / 30.44
  return {
    type: "time",
    label: null,
    ticks: spanMonths <= 3 ? d3.utcDay.every(7) : spanMonths <= 18 ? d3.utcMonth.every(1) : 8,
    tickFormat: d => {
      const m = d.getMonth(), y = d.getFullYear()
      if (spanMonths <= 18) return m === 0 ? `${MON[m]}-${y}` : MON[m]
      return m === 0 ? String(y) : m === 6 ? `Jul ${y}` : ""
    }
  }
})()

fmtN = d => d.toLocaleString()

System-Wide Occupancy — All Programs

Plot.plot({
  width, height: 400, marginLeft: 70,
  x: xAxis,
  y: {label: "Individuals accommodated", grid: true, tickFormat: fmtN},
  marks: [
    // Threshold bands (toggleable) — area between expected and threshold levels
    showBands.includes("2 MAD-SD")
      ? Plot.areaY(systemData, {x: "dateObj", y1: "expected", y2: "threshold_2mad",
          fill: "#ff9500", fillOpacity: 0.20})
      : null,
    showBands.includes("3 MAD-SD")
      ? Plot.areaY(systemData, {x: "dateObj", y1: "threshold_2mad", y2: "threshold_3mad",
          fill: "#ff3b30", fillOpacity: 0.15})
      : null,
    // Expected line (trend + seasonal)
    Plot.line(systemData, {x: "dateObj", y: "expected",
      stroke: "#aaa", strokeWidth: 1.5, strokeDasharray: "4,3"}),
    // Observed line
    Plot.line(systemData, {x: "dateObj", y: "observed",
      stroke: "#0066cc", strokeWidth: 2}),
    // Spike dots — colored by severity
    Plot.dot(systemData.filter(d => d.spike_flag !== "none"), {
      x: "dateObj",
      y: "observed",
      fill: d => d.spike_flag === "3mad" ? "#ff3b30" : "#ff9500",
      r: 4,
      title: d => `${d.date}\n${d.observed.toLocaleString()} individuals\n${d.remainder_mad_sd.toFixed(1)} MAD-SD above expected`
    }),
    Plot.tip(systemData, Plot.pointerX({
      x: "dateObj", y: "observed",
      title: d => {
        const flag = d.spike_flag === "3mad" ? " ★ SPIKE (3 MAD-SD)"
                   : d.spike_flag === "2mad" ? " ▲ Spike (2 MAD-SD)" : ""
        return `${d.date}${flag}\nObserved: ${d.observed.toLocaleString()}\nExpected: ${Math.round(d.expected).toLocaleString()}\nDeviation: ${d.remainder_mad_sd.toFixed(1)} MAD-SD`
      }
    }))
  ]
})

Men’s Sector — Most Affected

heroStats = {
  const sectors = ["men","women","mixed_adult","youth","families"]
  const menRows = spikeData.filter(d => d.series === "men")
  const menSpikes = menRows.filter(d => d.spike_flag !== "none").length
  const menSevere = menRows.filter(d => d.spike_flag === "3mad").length
  const totalSpikes = spikeData.filter(d => sectors.includes(d.series) && d.spike_flag !== "none").length
  const pct = Math.round(menSpikes / totalSpikes * 100)
  const startYear = spikeData.reduce((mn, d) => Math.min(mn, +d.date.slice(0,4)), 9999)
  return {menSpikes, menSevere, totalSpikes, pct, startYear}
}

html`<div class="crisis-hero">
  <div class="hero-number">${heroStats.pct}%</div>
  <div class="hero-text">
    <div class="hero-label">of all sector spike days in Toronto's shelter system occurred in the <strong>Men's sector</strong> — more than any other, and nearly double the next highest.</div>
    <div class="hero-sub">${heroStats.menSpikes.toLocaleString()} of ${heroStats.totalSpikes.toLocaleString()} total spike days &nbsp;·&nbsp; ${heroStats.menSevere} were severe (3 MAD-SD) &nbsp;·&nbsp; ${heroStats.startYear}–present</div>
  </div>
</div>`

_sectorLabels = ({men: "Men", women: "Women", mixed_adult: "Mixed Adult", youth: "Youth", families: "Families"})

sectorStats = ["men", "youth", "families", "women", "mixed_adult"].map(sec => {
  const rows = spikeData.filter(d => d.series === sec)
  const spikes = rows.filter(d => d.spike_flag !== "none").length
  const spikes3 = rows.filter(d => d.spike_flag === "3mad").length
  return {
    label: _sectorLabels[sec],
    spikes,
    spikes3,
    pctDays: spikes / rows.length * 100,
    isMen: sec === "men"
  }
}).sort((a, b) => b.spikes - a.spikes)

Plot.plot({
  width, height: 210, marginLeft: 110, marginRight: 160,
  x: {label: "% of days that were spike days", tickFormat: d => d + "%", domain: [0, 18]},
  y: {label: null},
  marks: [
    Plot.barX(sectorStats, {
      x: "pctDays", y: "label",
      fill: d => d.isMen ? "#ff3b30" : "#4a6fa5",
      fillOpacity: d => d.isMen ? 1 : 0.45,
      sort: {y: "-x"}
    }),
    Plot.text(sectorStats, {
      x: "pctDays", y: "label",
      text: d => `${d.pctDays.toFixed(1)}%  (${d.spikes} days, ${d.spikes3} severe)`,
      dx: 8, textAnchor: "start", fontSize: 13,
      fill: d => d.isMen ? "#ff3b30" : "#333",
      fontWeight: d => d.isMen ? "700" : "400"
    }),
    Plot.ruleX([0])
  ]
})

html`<p>Men are both the largest sector in Toronto's shelter system and the most frequently affected by occupancy spikes. Across ${heroStats.startYear}–present, ${heroStats.menSpikes.toLocaleString()} of the ${heroStats.totalSpikes.toLocaleString()} total sector spike days occurred in the Men's sector — ${heroStats.pct}%, more than any other sector by a significant margin. And unlike other sectors, Men's spike days skew toward the most severe band: ${heroStats.menSevere} of the ${heroStats.menSpikes.toLocaleString()} crossed the 3 MAD-SD threshold, compared to just ${heroStats.menSpikes - heroStats.menSevere} at the 2 MAD-SD level, indicating that Men's sector anomalies tend to be pronounced rather than subtle. This is consistent with a broader pattern in Toronto's homelessness data: men make up the majority of the city's chronically homeless population. When the system comes under pressure, the Men's sector shows it first and most sharply. That pattern is visible here.</p>`

Plot.plot({
  width, height: 400, marginLeft: 70,
  x: xAxis,
  y: {label: "Men — individuals accommodated", grid: true, tickFormat: fmtN},
  marks: [
    showBands.includes("2 MAD-SD")
      ? Plot.areaY(menData, {x: "dateObj", y1: "expected", y2: "threshold_2mad",
          fill: "#ff9500", fillOpacity: 0.20})
      : null,
    showBands.includes("3 MAD-SD")
      ? Plot.areaY(menData, {x: "dateObj", y1: "threshold_2mad", y2: "threshold_3mad",
          fill: "#ff3b30", fillOpacity: 0.15})
      : null,
    Plot.line(menData, {x: "dateObj", y: "expected",
      stroke: "#aaa", strokeWidth: 1.5, strokeDasharray: "4,3"}),
    Plot.line(menData, {x: "dateObj", y: "observed",
      stroke: "#0066cc", strokeWidth: 2}),
    Plot.dot(menData.filter(d => d.spike_flag !== "none"), {
      x: "dateObj", y: "observed",
      fill: d => d.spike_flag === "3mad" ? "#ff3b30" : "#ff9500",
      r: 4,
      title: d => `${d.date}\n${d.observed.toLocaleString()} individuals\n${d.remainder_mad_sd.toFixed(1)} MAD-SD above expected`
    }),
    Plot.tip(menData, Plot.pointerX({
      x: "dateObj", y: "observed",
      title: d => {
        const flag = d.spike_flag === "3mad" ? " ★ SPIKE (3 MAD-SD)"
                   : d.spike_flag === "2mad" ? " ▲ Spike (2 MAD-SD)" : ""
        return `Men${flag}\n${d.date}\nObserved: ${d.observed.toLocaleString()}\nExpected: ${Math.round(d.expected).toLocaleString()}\nDeviation: ${d.remainder_mad_sd.toFixed(1)} MAD-SD`
      }
    }))
  ]
})

Other Sectors

Each panel uses its own y-axis and its own threshold — sectors differ too much in size to share a scale. When the system-wide or Men’s chart shows a spike, these panels show whether other sectors were also affected.

// Small multiples — one Plot.plot() per sector
// Use html`` template to arrange in CSS grid
html`<div class="sector-grid">${sectorOrder.map(sec => {
  const secData = spikeData
    .filter(d => d.series === sec && d.dateObj >= windowCutoff)
    .sort((a, b) => a.dateObj - b.dateObj)
  const label = sectorLabels[sec]
  if (!secData.length) return html`<div class="sector-panel"><em>${label}: no data</em></div>`
  const chart = Plot.plot({
    width: Math.max(300, Math.floor((width - 40) / 3)),
    height: 220,
    marginLeft: 60,
    marginTop: 30,
    caption: label,
    x: xAxis,
    y: {label: null, grid: true, tickFormat: fmtN},
    marks: [
      showBands.includes("2 MAD-SD")
        ? Plot.areaY(secData, {x: "dateObj", y1: "expected", y2: "threshold_2mad",
            fill: "#ff9500", fillOpacity: 0.20})
        : null,
      showBands.includes("3 MAD-SD")
        ? Plot.areaY(secData, {x: "dateObj", y1: "threshold_2mad", y2: "threshold_3mad",
            fill: "#ff3b30", fillOpacity: 0.15})
        : null,
      Plot.line(secData, {x: "dateObj", y: "expected",
        stroke: "#aaa", strokeWidth: 1, strokeDasharray: "3,2"}),
      Plot.line(secData, {x: "dateObj", y: "observed",
        stroke: "#0066cc", strokeWidth: 1.5}),
      Plot.dot(secData.filter(d => d.spike_flag !== "none"), {
        x: "dateObj", y: "observed",
        fill: d => d.spike_flag === "3mad" ? "#ff3b30" : "#ff9500",
        r: 3,
        title: d => `${d.date}\n${d.observed.toLocaleString()} individuals\n${d.remainder_mad_sd.toFixed(1)} MAD-SD`
      }),
      Plot.tip(secData, Plot.pointerX({
        x: "dateObj", y: "observed",
        title: d => {
          const flag = d.spike_flag === "3mad" ? " ★ SPIKE"
                     : d.spike_flag === "2mad" ? " ▲ Spike" : ""
          return `${label}${flag}\n${d.date}\n${d.observed.toLocaleString()} individuals`
        }
      }))
    ]
  })
  return html`<div class="sector-panel">${chart}</div>`
})}</div>`

Spike Days

// All spike rows across all series, most-recent-first
allSpikes = spikeData
  .filter(d => d.spike_flag !== "none")
  .sort((a, b) => b.date.localeCompare(a.date))
  .map(d => ({
    Date: d.date,
    Metric: d.label,
    Observed: d.observed,
    Expected: Math.round(d.expected),
    "Deviation (MAD-SD)": +d.remainder_mad_sd.toFixed(1),
    Band: d.spike_flag === "3mad" ? "3 MAD-SD ★" : "2 MAD-SD",
    Notes: ""
  }))

viewof spikeSearch = Inputs.search(allSpikes, {placeholder: "Filter by metric or date…"})

Inputs.table(spikeSearch, {
  columns: ["Date","Metric","Observed","Expected","Deviation (MAD-SD)","Band","Notes"],
  format: {
    Observed: d => d.toLocaleString(),
    Expected: d => d.toLocaleString()
  },
  sort: "Date",
  reverse: true
})

---

Contains information licensed under the Open Government Licence – Toronto. Source: City of Toronto Open Data — Daily Shelter & Overnight Service Occupancy & Capacity. Analysis by BonQuery.