index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Sol Messing</title>
    <link>https://solmessing.netlify.app/</link>
      <atom:link href="https://solmessing.netlify.app/index.xml" rel="self" type="application/rss+xml" />
    <description>Sol Messing</description>
    <generator>Wowchemy (https://wowchemy.com)</generator><language>en-us</language><lastBuildDate>Mon, 24 Oct 2022 00:00:00 +0000</lastBuildDate>
    <image>
      <url>https://solmessing.netlify.app/media/icon_huedd5de82286bc1d0a1509b535f624c76_25974_512x512_fill_lanczos_center_3.png</url>
      <title>Sol Messing</title>
      <link>https://solmessing.netlify.app/</link>
    </image>
    
    <item>
      <title>Example Talk</title>
      <link>https://solmessing.netlify.app/talk/example-talk/</link>
      <pubDate>Sat, 01 Jun 2030 13:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/talk/example-talk/</guid>
      <description>&lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Click on the &lt;strong&gt;Slides&lt;/strong&gt; button above to view the built-in slides feature.
  &lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Slides can be added in a few ways:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Create&lt;/strong&gt; slides using Wowchemy&amp;rsquo;s &lt;a href=&#34;https://wowchemy.com/docs/managing-content/#create-slides&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Slides&lt;/em&gt;&lt;/a&gt; feature and link using &lt;code&gt;slides&lt;/code&gt; parameter in the front matter of the talk file&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Upload&lt;/strong&gt; an existing slide deck to &lt;code&gt;static/&lt;/code&gt; and link using &lt;code&gt;url_slides&lt;/code&gt; parameter in the front matter of the talk file&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Embed&lt;/strong&gt; your slides (e.g. Google Slides) or presentation video on this page using &lt;a href=&#34;https://wowchemy.com/docs/writing-markdown-latex/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;shortcodes&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Further event details, including &lt;a href=&#34;https://wowchemy.com/docs/writing-markdown-latex/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;page elements&lt;/a&gt; such as image galleries, can be added to the body of this page.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>An Early Election 2024 Forecast</title>
      <link>https://solmessing.netlify.app/post/election_projection_regularized_swing/</link>
      <pubDate>Thu, 11 Jan 2024 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/election_projection_regularized_swing/</guid>
      <description>&lt;p&gt;Early projections for 2024 based on previous Presidential and House returns slighly favor Republicans. These projections are completely unrelated to Biden’s recent polling numbers.&lt;/p&gt;
&lt;!-- ![](/img/Map_for_2024_JS_Swing.png &#34;A simple forecasted map for 2024. Created https://www.270towin.com/maps/WWE2B.&#34;) --&gt;
&lt;p&gt;Here&amp;rsquo;s the story behind this approach: In early 2020, I ran battleground state election forecasts for Acronym. The results suggested Georgia would be extremely competitive—and Acronym spent more $ there than many other non-profit actors. After the election, we could see that those projections had much lower forecasting error than polling data &lt;a href=&#34;https://solomonmg.github.io/post/what-the-polls-got-wrong-in-2020/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://solomonmg.github.io/post/what-the-polls-got-wrong-in-2020/&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Because this approach does not use polling data, it&amp;rsquo;s not suspetible to any of the potential problems with polls I talk about in that post: undecided voters breaking late, low education non-response, bad likely voter modeling, partisan non-response, shy Trumpers, etc.&lt;/p&gt;
&lt;p&gt;The core idea behind this approach is a fact not emphasized enough in most stats/ML courses: if you’re going to try to predict something, it’s very hard to do better than using the same variable at t - 1 if you can. And we can. This approach goes one step further and looks at the direction that variable has been moving and assume that things are likely to keep moving in that same direction.&lt;/p&gt;
&lt;p&gt;What that means for presidential election forecasts: for each state, estimate the &amp;ldquo;swing&amp;rdquo; from 2016 to 2020 for president and 2018-2022 for the U.S. house; then simply add that to 2020 presidential returns. Then those estimates of state-level swing are regularized&amp;mdash;mathematically ``nudged&amp;rsquo;&amp;rsquo; toward national trends, which you’ll like if you believe “uniform swing” is particularly important. The projected state-level swing is weighted 60-40 toward presidential results.&lt;/p&gt;
&lt;p&gt;Here&amp;rsquo;s a cleaner plot showing the actual forecast values in potential 2024 battleground states:&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-a-simple-forecast-for-2024-battleground-states-hat-tip-to-tom-cunninghamhttpstecunninghamgithubio-who-suggested-this-plot-design&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/EstStateDemVote.png&#34; alt=&#34;A simple forecast for 2024 battleground states. Hat tip to [Tom Cunningham](https://tecunningham.github.io) who suggested this plot design.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      A simple forecast for 2024 battleground states. Hat tip to &lt;a href=&#34;https://tecunningham.github.io&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tom Cunningham&lt;/a&gt; who suggested this plot design.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;I should now point to a link to the data and code: &lt;a href=&#34;https://github.com/SolomonMg/election_projection_regularized_swing&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://github.com/SolomonMg/election_projection_regularized_swing&lt;/a&gt;, and thank the &lt;a href=&#34;https://electionlab.mit.edu&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;MIT Election Data + Science Lab&lt;/a&gt; for curating &lt;a href=&#34;https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/42MVDX&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;these&lt;/a&gt; &lt;a href=&#34;https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/IG0UN2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;data&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;electoral-math&#34;&gt;Electoral Math:&lt;/h3&gt;
&lt;p&gt;I&amp;rsquo;m going to rely on &lt;a href=&#34;https://www.270towin.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;www.270towin.com&lt;/a&gt; to translate these projections into an electoral map. A better way to do this might be to come up with conservative estimates of error and simulate a few thousand elections, but I&amp;rsquo;m not estimating an extremely rigorous Bayesian model nor including enough extant data to really justify a FiveThirtyEight style forecast.&lt;/p&gt;
&lt;p&gt;If you call anything lower than a 3 point margin either way a &amp;ldquo;tossup,&amp;rdquo; here&amp;rsquo;s what the electoral map looks like:&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-elecotral-map-for-2024-lower-than-a-3-margin-is-a-tossup-created-httpswww270towincommapswwe2b&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/Map_for_2024_JS_Swing.png&#34; alt=&#34;Elecotral map for 2024, lower than a 3% margin is a tossup. Created https://www.270towin.com/maps/WWE2B.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Elecotral map for 2024, lower than a 3% margin is a tossup. Created &lt;a href=&#34;https://www.270towin.com/maps/WWE2B&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://www.270towin.com/maps/WWE2B&lt;/a&gt;.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;That looks OK for Biden, but if you really trust this approach, you might want to say anything lower than 2% is a tossup. Then the electoral math looks very bad for Biden:&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-elecotral-map-for-2024-lower-than-a-2-margin-is-a-tossup-created-httpswww270towincommapswwexg&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/Map_for_2024_JS_margin2Swing.png&#34; alt=&#34;Elecotral map for 2024, lower than a 2% margin is a tossup. Created https://www.270towin.com/maps/WWExg.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Elecotral map for 2024, lower than a 2% margin is a tossup. Created &lt;a href=&#34;https://www.270towin.com/maps/WWExg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://www.270towin.com/maps/WWExg&lt;/a&gt;.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h3 id=&#34;observation-polarization-and-accuracy&#34;&gt;Observation: Polarization and Accuracy&lt;/h3&gt;
&lt;p&gt;These projections essentially assume party identification, demographic trends, and voting behavior will mostly continue in the same general direction as in the past. They should have a lot of appeal if you think polarization means most people have already made up their minds about who to vote for for President, that Presidential campaign effects are relatively small (in equilibrium at least), and/or that “demographics are destiny.” What&amp;rsquo;s more, the results are regularized toward national trends, which you’ll like if you believe that &lt;a href=&#34;https://press.uchicago.edu/ucp/books/book/chicago/I/bo27596045.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;local politics has been ``nationalized,&amp;rsquo;&amp;rsquo; as Dan Hopkins argues&lt;/a&gt; and thus that &lt;a href=&#34;https://projects.fivethirtyeight.com/2020-swing-states/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;“uniform swing” in the electorate is an increasingly important factor explaining state-level election results&lt;/a&gt;&amp;mdash;despite that Florida bucked the national trend in 2020.&lt;/p&gt;
&lt;p&gt;In fact, over time, as polarization seems to worsen, this approach improves in accuracy:&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-backtested-forecasts-improve-in-accuracy-over-time-and-2020-was-far-easier-to-predict-than-past-elections&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/Battleground_MAE_projections.png&#34; alt=&#34;Backtested forecasts improve in accuracy over time, and 2020 was far easier to predict than past elections.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Backtested forecasts improve in accuracy over time, and 2020 was far easier to predict than past elections.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;However, these projections do not account for events since 2022. Older voters pass away and younger voters become eligible to vote changing the makeup of the electorate. Public opinion/sentiment may change related to economic conditions (inflation/income/unemployment/etc), policy developments e.g., related to abortion, international affairs like the Gaza conflict, or candidate-attributes like Biden’s age or Trumps legal troubles.&lt;/p&gt;
&lt;h3 id=&#34;observation-patterns-in-us-elections&#34;&gt;Observation: Patterns in U.S. Elections&lt;/h3&gt;
&lt;p&gt;These projections also do not explicitly model well-known voting patterns, instead relying on change from one cycle to another to get reasonable estimates. The most notable trend is that the president’s party almost always tends to lose seats in the house in midterm elections. &lt;a href=&#34;https://www.jstor.org/stable/2130810&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://www.jstor.org/stable/2130810&lt;/a&gt; &lt;a href=&#34;https://fivethirtyeight.com/features/why-the-presidents-party-almost-always-has-a-bad-midterm/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://fivethirtyeight.com/features/why-the-presidents-party-almost-always-has-a-bad-midterm/&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Because the model only looks at the state-level the difference between the last two &lt;em&gt;midterm&lt;/em&gt; cycles, these projections are capturing how midterm returns &lt;em&gt;change&lt;/em&gt; in each state, which goes a ways toward correcting the consistent lower performance in midterms pattern, and in part may reflect changes in sentiment toward the president.&lt;/p&gt;
&lt;p&gt;A less reliable trend that’s held since FDR’s time is that incumbent presidents have tended to get a higher percent of the popular vote in their election for a second term&amp;mdash;Obama in 2012 was a notable exception. &lt;a href=&#34;https://www.presidency.ucsb.edu/statistics/data/presidential-election-mandates&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://www.presidency.ucsb.edu/statistics/data/presidential-election-mandates&lt;/a&gt; What’s more, house midterm results seem to be particularly bad just before an incumbent is voted out of office. It&amp;rsquo;s not clear if this is a bug or a feature, or how reliably this would be picked up using this method, but it&amp;rsquo;s worthing pointing out.&lt;/p&gt;
&lt;h3 id=&#34;observation-accuracy-over-previous-election-results&#34;&gt;Observation: Accuracy over Previous Election Results&lt;/h3&gt;
&lt;p&gt;If it&amp;rsquo;s hard to do better than election returns at t - 1, does this approach actually do better? Yes, by a little. Including all states, these projections have lower mean absolute error (MAE). For some reason these projections miss badly in 2004, and excluding that earliest year I can compute these projections using the MIT data, shows they do in fact do quite a bit better than simply relying on previous presidential election results alone.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-accuracy-over-time-for-projections-compared-with-simply-using-the-previous-election&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/AllStates_MAE_projections.png&#34; alt=&#34;Accuracy over time for projections compared with simply using the previous election. &#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Accuracy over time for projections compared with simply using the previous election.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;bt_dat&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;group_by&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;proj_type&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;summarise&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mae&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# A tibble: 2 × 2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;n&#34;&gt;proj_type&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;`mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mae&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;`
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;  &amp;lt;chr&amp;gt;           &amp;lt;dbl&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;1 prev pres        3.11
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;2 proj             2.65
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;&amp;gt; bt_dat %&amp;gt;% filter(years != 2004) %&amp;gt;% group_by(proj_type) %&amp;gt;% summarise(mean(mae))
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;# A tibble: 2 × 2
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;  proj_type `&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mae&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;`&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;chr&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;&lt;/span&gt;           &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;dbl&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;prev&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;        &lt;span class=&#34;m&#34;&gt;3.36&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;proj&lt;/span&gt;             &lt;span class=&#34;m&#34;&gt;2.51&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Results are more subtle if we restrict to battleground states:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;bt_dat&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;group_by&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;proj_type&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;summarise&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mae&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# A tibble: 2 × 2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;n&#34;&gt;proj_type&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;`mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mae&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;`
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;  &amp;lt;chr&amp;gt;           &amp;lt;dbl&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;1 prev pres        1.19
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;2 proj             1.16
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;&amp;gt; bt_dat %&amp;gt;% filter(years != 2004) %&amp;gt;% group_by(proj_type) %&amp;gt;% summarise(mean(mae))
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;# A tibble: 2 × 2
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;  proj_type `&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mae&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;`&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;chr&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;&lt;/span&gt;           &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;dbl&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;prev&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;       &lt;span class=&#34;m&#34;&gt;1.28&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;proj&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;0.995&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;observation-regularization-toward-0-or-the-mean&#34;&gt;Observation: Regularization toward 0 or the Mean?&lt;/h3&gt;
&lt;p&gt;Here’s the map with shrinkage toward 0, which will move the estimates toward the prior year&amp;rsquo;s election. Biden does worse in WI and slightly worse in AZ and PA, because the presidential swing estimates get pulled down toward zero instead of up toward the nation-wide state-level mean (3.5%). But he does better in NC, where the relatively good house results get pulled toward zero instead of down to the state-level average midterm swing (-11.5%).&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-a-simple-forecasted-map-for-2024-shrinkage-toward-zero-created-httpswww270towincommaps66rzw&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/Map_for_2024_JS_0_Swing.png&#34; alt=&#34;A simple forecasted map for 2024, shrinkage toward zero. Created https://www.270towin.com/maps/66rZw.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      A simple forecasted map for 2024, shrinkage toward zero. Created &lt;a href=&#34;https://www.270towin.com/maps/66rZw&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://www.270towin.com/maps/66rZw&lt;/a&gt;.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;However, based on my updated backtesting, the MAE estimates are worse when you shrink toward zero, which is what I did back in 2020. This makes me feel good because the mathematical/statistical theory says that shrinking toward the group mean should produce high quality estimates, while there&amp;rsquo;s not much theory that suggests shrinking toward zero should improve estimation.&lt;/p&gt;
&lt;h3 id=&#34;methological-details&#34;&gt;Methological Details&lt;/h3&gt;
&lt;p&gt;For each state it estimates the &amp;ldquo;swing&amp;rdquo; from 2016 to 2020 for president and 2018-2022 for the U.S. house; then simply adds that to 2020 presidential returns. The projected state-level swing is weighted 60-40 toward presidential results.&lt;/p&gt;
&lt;p&gt;Now, the tricky bit is I estimate “swing” using James-Stein-adjusted state-level slope. This method “shrinks” the slant of each slope toward 50-50 or toward the average slope. 50-50 is what I used in 2020 and but recent corrections I&amp;rsquo;ve made to my backtesting scrips reveals that has a slightly higher mean absolute error going back to the 2004 election.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;ve since updated the approach in a number of important ways, based on backtesting (looking at how well the method performs on past elections). I now regularize (or &amp;ldquo;shrink&amp;rdquo;) fewer quantities and do so toward the mean instead of toward zero. I should also note that the original code I used a few minor errors, which I&amp;rsquo;ve since fixed.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Disaggregating &#39;Ideological Segregation&#39;</title>
      <link>https://solmessing.netlify.app/post/thoughts-on-election-2020/</link>
      <pubDate>Wed, 02 Aug 2023 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/thoughts-on-election-2020/</guid>
      <description>&lt;h3 id=&#34;tldr&#34;&gt;TLDR:&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;[UPDATED SEPT 30] Yesterday, Science &lt;a href=&#34;https://solmessing.netlify.app/pdf/science.adk1211.pdf&#34;&gt;published a letter I wrote&lt;/a&gt; arguing that there is little evidence of algorithmic bias in Facebook’s feed ranking system that would serve to increase ideological segregation, also known as &lt;a href=&#34;https://books.google.com/books/about/The_Filter_Bubble.html?id=Qn2ZnjzCE3gC&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;the &amp;ldquo;Filter Bubble&amp;rdquo; hypothesis&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;This contradicts claims in &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; that Newsfeed ranking increases ideological segregation. This claim was the main piece of evidence in the Science &lt;a href=&#34;https://www.science.org/toc/science/381/6656&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Special Issue on Meta&lt;/a&gt; that might support the controversial cover that suggested that Meta’s algorithms are “Wired to Split.”&lt;/li&gt;
&lt;li&gt;The issue is that while domain-level analysis suggests feed-ranking increases ideological segregation, URL-level analysis shows &lt;em&gt;no difference&lt;/em&gt; in ideological segregation before and after feed-ranking.&lt;/li&gt;
&lt;li&gt;And we should strongly prefer their URL-level analysis. Domain-level analysis &lt;em&gt;effectively mislabels highly partisan content&lt;/em&gt; as &amp;ldquo;moderate/mixed,&amp;rdquo; especially on websites like YouTube, Reddit, and Twitter (&lt;a href=&#34;https://ori.hhs.gov/education/products/niu_authorship/mistakes/09mistake-a.htm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;aggregation bias/ecological fallacy&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;Interestingly, the authors seem to agree&amp;mdash;the discussion section points out problems with domain-level analysis.&lt;/li&gt;
&lt;li&gt;Another &lt;em&gt;Science&lt;/em&gt; paper from the same issue, &lt;a href=&#34;https://www.science.org/doi/10.1126/science.abp9364&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Guess et al 2023&lt;/a&gt; shows (in the SM) that Newsfeed ranking actually &lt;em&gt;decreases&lt;/em&gt; exposure to &lt;em&gt;political content&lt;/em&gt; from like-minded sources compared with reverse-chronological feedranking.&lt;/li&gt;
&lt;li&gt;The evidence in the 4 recent papers is not consistent with a meaningful Filter Bubble effect in 2020; nor does it support the notion that Meta&amp;rsquo;s algorithms are &amp;ldquo;Wired to Split.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Furthermore, domain-level aggregation bias is a big issue in a great deal of past research on ideological segregation, because domain-level analysis &lt;em&gt;understates&lt;/em&gt; media polarization. Because &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; gives both URL- and domain-level estimates, we can see the magnitude of aggregation bias. It&amp;rsquo;s huge.&lt;/li&gt;
&lt;li&gt;I make a number of other observations about what we know about whether social media is polarizing and discuss implications for the controversial Science cover and Meta&amp;rsquo;s flawed claims that this research is exculpatory.&lt;/li&gt;
&lt;/ol&gt;
&lt;!-- 6. None of this is the last word on social media algorithms---as [González-Bailón et al 2023](https://www.science.org/doi/full/10.1126/science.ade7138) point out, we need additional research on friend/page/group/etc recommender systems, which may polarize the graph itself.  --&gt;
&lt;h3 id=&#34;introicymi&#34;&gt;Intro/ICYMI&lt;/h3&gt;
&lt;details&gt;
  &lt;summary&gt;Click to expand&lt;/summary&gt;
&lt;p&gt;Last week saw the release of a series of &lt;a href=&#34;https://www.science.org/toc/science/381/6656&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;excellent papers in &lt;em&gt;Science&lt;/em&gt;&lt;/a&gt;. I was particularly interested in &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt;, which measures &amp;ldquo;ideological segregation.&amp;rdquo; This concept is based on &lt;a href=&#34;https://web.stanford.edu/~gentzkow/research/echo_chambers.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Matt Gentzkow and Jesse Shapiro&amp;rsquo;s 2011 work&lt;/a&gt;. As they note, &amp;ldquo;The index ranges from 0 (all conservative and liberal visits are to the same outlet) to 1 (conservatives only visit 100% conservative outlets and liberals only visit 100% liberal outlets).&amp;rdquo;&lt;/p&gt;
&lt;p&gt;This paper also replicates and extends my own work with Eytan Bakshy and Lada Adamic, also published in &lt;a href=&#34;https://solomonmg.github.io/pdf/Science-2015-Bakshy-1130-2.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Science&lt;/em&gt; in 2015&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;To be clear &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; goes a lot further, examining how these factors vary over-time, investigating clusters of isolated partisan media organizations, and patterns in the consumption of misinformation. They find (1) ideological segregation is high; (2) ideological segregation “increases after algorithmic curation” consistent with the “Filter Bubble” hypothesis; (3) there is a substantial right wing “echo chamber” in which conservatives are essentially siloed from the rest of the site (4) where misinformation thrives.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;ve had many years to think about issues related to these questions, after working with similar data from 2012 in my dissertation, and co-authoring a &lt;a href=&#34;https://solomonmg.github.io/pdf/Science-2015-Bakshy-1130-2.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Science paper&lt;/a&gt; while working at Facebook using 2014 data. I also saw the design (but not results) presented at the &lt;a href=&#34;https://www.ssrc.org/programs/digital-platforms-initiative/2023-ssrc-workshop-on-the-economics-of-social-media/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;2023 SSRC Workshop on the Economics of Social Media&lt;/a&gt;, though I did not notice these issues until I saw the final paper.&lt;/p&gt;
&lt;p&gt;I put together these thoughts after discussion and feedback from Dean Eckles and Tom Cunningham, former colleagues at Stanford, Facebook, and Twitter.&lt;/p&gt;
&lt;!-- [Click here to read the backstory on Echo Chambers, Filter Bubbles, and Selective Exposure, including where my own past work fits in](#a-brief-history-of-echo-chambers-filter-bubbles-and-selective-exposure) --&gt;
&lt;/details&gt;
&lt;h3 id=&#34;a-brief-history-of-echo-chambers-filter-bubbles-and-selective-exposure&#34;&gt;A Brief History of Echo Chambers, Filter Bubbles, and Selective Exposure&lt;/h3&gt;
&lt;details&gt;
  &lt;summary&gt;Click to expand&lt;/summary&gt;
&lt;!-- 20 years ago I worked as a foreign media analyst and I noticed that a great deal of misinformation circulating in news websites in Indonesia and the Middle East came from Alex Jones&#39; InfoWars (yes he&#39;s been around for a long time). I became fascinated with the question of how people get their media and how technology changes that.  --&gt;
&lt;p&gt;The conventional academic wisdom when I started my PhD was that we shouldn&amp;rsquo;t expect to see much in the way of media effects (&lt;a href=&#34;https://books.google.com/books/about/The_Effects_of_Mass_Communication.html?id=CzcGAQAAIAAJ&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Klapper 1960&lt;/a&gt;) because people tended to &amp;ldquo;select into&amp;rdquo; content that reinforced their views (&lt;a href=&#34;https://www.jstor.org/stable/2747198&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Sears and Freedman 1967&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;In 2007, Cass Sunstein wrote &lt;a href=&#34;https://www.jstor.org/stable/j.ctt7tbsw&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;Republic.com 2.0&amp;rdquo;&lt;/a&gt;, which warned that the internet could allow us to even more easily isolate ourselves into &amp;ldquo;information cocoons&amp;rdquo; and &amp;ldquo;echo chambers.&amp;rdquo; Technology allows us to &amp;ldquo;filter&amp;rdquo; exactly what we want to see, and design our own programming. Cass also suggested this could lead to polarization.&lt;/p&gt;
&lt;!-- What&#39;s more, new media should be expected to further strengthen this &#34;minimal effects&#34; hypothesis ([Bennett and Iyengar 2008](https://citeseerx.ist.psu.edu/document?repid=rep1&amp;type=pdf&amp;doi=a2fcebd37ba1e919f662a287841b0356603cd0a4)).  --&gt;
&lt;p&gt;In 2009-2010, Sean Westwood and I ran a series of studies suggesting that &lt;a href=&#34;https://journals.sagepub.com/doi/10.1177/0093650212466406&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;popularity and social cues&lt;/a&gt; in news aggregators and social media websites might be a way out of selective news consumption&amp;mdash;we might be &lt;em&gt;more&lt;/em&gt; exposed to cross-cutting views on platforms that feature a social component, and furthermore, this social component seemed to be more important that media &amp;ldquo;source&amp;rdquo; label. That was great in the abstract, but what happens on actual websites that people use?&lt;/p&gt;
&lt;p&gt;The extent to which widely used social media platforms might allow us to exit &amp;ldquo;echo chambers&amp;rdquo; depended on the extent of cross-partisan friendships and interactions on platforms like Facebook. I did a PhD internship at Facebook to look into that, and the question of whether encountering &lt;a href=&#34;https://www.dropbox.com/s/nu39148ukbab34r/CH7brief.pdf?raw=true&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;political news on social media was ideologically polarizing&lt;/a&gt; (note that the results are as not well powered as I would like). That work evolved into dissertation chapters and eventually our &lt;a href=&#34;https://solomonmg.github.io/pdf/Science-2015-Bakshy-1130-2.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Science paper&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Now &lt;a href=&#34;https://en.wikipedia.org/wiki/Eli_Pariser&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Eli Parsner&lt;/a&gt; had just published a book on &amp;ldquo;&lt;a href=&#34;https://books.google.com/books/about/The_Filter_Bubble.html?id=Qn2ZnjzCE3gC&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Filter Bubbles&lt;/a&gt;&amp;rdquo; suggesting that media technologies like Google Search and Facebook NewsFeed not only allowed us to ignore the &amp;ldquo;other side,&amp;rdquo; but actively filtering out search results and friends posts with perspectives from the other side.&lt;/p&gt;
&lt;p&gt;What&amp;rsquo;s more, a lot of people in the Human Computer Interaction (HCI) world were very interested in how one might examine this empirically, and I started collecting data with Eytan Bakshy that would do just that. The paper would allow us to quantify echo chambers created by our network of contacts, filter bubbles, and partisan selective exposure in social media by looking at exposure to &lt;a href=&#34;https://www.jstor.org/stable/3117813&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ideologically &amp;ldquo;cross-cutting&amp;rdquo; content&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;We defined a few key components:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Random&lt;/strong&gt; - The set of content (external URLs) shared on Facebook writ large.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Potential&lt;/strong&gt; - The set of content shared by one&amp;rsquo;s friends&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Exposed&lt;/strong&gt; - The set of content appearing in one&amp;rsquo;s Newsfeed.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Selected&lt;/strong&gt; - The set of content one clicks on.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Endorsed&lt;/strong&gt; - The set of content one &amp;rsquo;likes&#39;.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-figure-6-from-messing-2013-our-original-analysis-of-the-distribution-of-ideologically-aligned-content-on-facebook-using-data-from-2012&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/Ch6Fig6.5.jpg&#34; alt=&#34;Figure 6 from Messing 2013: Our original analysis of the distribution of ideologically-aligned content on Facebook using data from 2012.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Figure 6 from Messing 2013: Our original analysis of the distribution of ideologically-aligned content on Facebook using data from 2012.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-figure-3b-from-bakshy-et-al-2015-data-from-2014-quantifying-cross-cutting-content-on-facebook&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/ScienceBakshyFig3B.jpg&#34; alt=&#34;Figure 3B from Bakshy et al 2015: Data from 2014 quantifying cross-cutting content on Facebook.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Figure 3B from Bakshy et al 2015: Data from 2014 quantifying cross-cutting content on Facebook.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Unlike when I started to study social media in 2009 and no one was interested, in 2014, people understood that social media was an important force that was reshaping at least media if not society more broadly. And Science was particularly interested in the role of algorithms played in this environment.&lt;/p&gt;
&lt;p&gt;We published our results in &lt;a href=&#34;https://solomonmg.github.io/pdf/Science-2015-Bakshy-1130-2.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Science&lt;/a&gt;, and the response from many was &amp;ldquo;well this is smaller than expected,&amp;rdquo; including a piece in &lt;em&gt;Wired&lt;/em&gt; from &lt;a href=&#34;https://www.wired.com/2015/05/did-facebooks-big-study-kill-my-filter-bubble-thesis/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Eli Parisner himself&lt;/a&gt;. David Lazer, one of the lead authors of &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt;, also &lt;a href=&#34;https://education.biu.ac.il/sites/education/files/shared/science-2015-lazer-1090-1.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;wrote a perspective in &lt;em&gt;Science&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;However, the piece immediately drew a great deal of criticism. This is in part because I wrote that exposure was driven more by individual choices than algorithms, which ignored the potential influence of friend recommendation systems and, more broadly, swept aside the extent to which interfaces structure interactions on websites, which my own dissertation work had shown was quite substantial.&lt;/p&gt;
&lt;p&gt;The study also had important limitations, many of which I addressed in a post suggesting how &lt;a href=&#34;https://solomonmg.github.io/post/exposure-to-ideologically-diverse-response/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;future work could provide a more robust picture&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;I was (much later) tech lead for Social Science One (2018-2020), which gave external researchers access to data (the &lt;a href=&#34;https://solmessing.netlify.app/pdf/Facebook_DP_URLs_Dataset.pdf&#34;&gt;&amp;lsquo;Condor&amp;rsquo; URLs data set&lt;/a&gt;) via differential privacy. My goal was to enable the kind of work research done &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt;. However, it soon became clear that Social Science One&amp;rsquo;s data sharing model and differential privacy in particular was not suitable for ground-breaking research.&lt;/p&gt;
&lt;p&gt;I personally advocated (with Facebook Researcher and longtime colleague &lt;a href=&#34;https://twitter.com/anniefranco&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Annie Franco&lt;/a&gt;) for the collaboration model used in the Election 2020 project, wherein external researchers would collaborate with Facebook researchers. That model would have to shield the research from any interference from Facebook&amp;rsquo;s Communications and Policy arm, which might attempt to interfere with the inturpretation or publication of any resulting papers, which would create ethical conflicts.&lt;/p&gt;
&lt;p&gt;I advocated for pre-registration to accomplish this, not merely to ensure scientific rigor but to protect against conflicts of interest and selective reporting of results. However, I left Facebook in January of 2020 and have not been deeply involved in the project since.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; does in fact accomplish most if not all of what I recommended future research do and goes much further than our original study, and the authors should be applauded for it. The paper shows that on Facebook (1) ideological segregation is high (in fact it&amp;rsquo;s arguably higher than implied in the paper); (2) there is a substantial right wing &amp;ldquo;echo chamber&amp;rdquo; in which conservatives are siloed from the rest of the site (3) where misinformation thrives. When they start to talk about the filter bubble though, things get more complicated.&lt;/p&gt;
&lt;/details&gt;
&lt;h3 id=&#34;is-there-a-filter-bubble-on-facebook&#34;&gt;Is there a Filter Bubble on Facebook&lt;/h3&gt;
&lt;p&gt;Are Facebook&amp;rsquo;s algorithms &amp;ldquo;Wired to split&amp;rdquo; the public? This question is at the core of the recent &lt;em&gt;Science&lt;/em&gt; issue, it&amp;rsquo;s hotly debated in the field of algorithmic bias. A suspicion that the answer is &amp;ldquo;yes&amp;rdquo; has motivated a number of policy and regulatory actions. Armed with unprecendented data, &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; seeks to answer this and other related questions.&lt;/p&gt;
&lt;p&gt;There are three relevant claims that answer this question in the text: (1) &amp;ldquo;ideological segregation is high and increases as we shift from potential exposure to actual exposure to engagement&amp;rdquo; in the abstract, (2) &amp;ldquo;The algorithmic promotion of compatible content from this inventory is positively associated with an increase in the observed segregation as we move from potential to exposed audiences&amp;rdquo; in the discussion section, and (3) &amp;ldquo;Segregation scores drawn from exposed audiences are higher than those based on potential audiences &amp;hellip; (the difference between potential and engaged audiences is only visible at the domain level),&amp;rdquo; in the caption of Figure 2.&lt;/p&gt;
&lt;p&gt;These statements are generally confirmatory of algorithmic segregation, aka the &lt;a href=&#34;https://books.google.com/books/about/The_Filter_Bubble.html?id=Qn2ZnjzCE3gC&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Filter Bubble hypothesis&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;But look at Figure 2, on which these claims seem to be based. Figure 2B shows an increase in observed segregation as you move from potential to exposed audiences. BUT Figure 2C&amp;mdash;describing the same phenomena&amp;mdash;does &lt;em&gt;not&lt;/em&gt; (as noted in the caption).&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-when-viewed-at-the-level-of-the-url-2c-the-study-is-consistent-with-a-negligible-filter-bubble-effect&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BailonFig2BC.jpeg&#34; alt=&#34;Figure 2 (B &amp;amp;amp; C) from González-Bailón et al 2023: When viewed at the level of the URL (2C), the study is consistent with a &amp;lt;em&amp;gt;negligible&amp;lt;/em&amp;gt; Filter Bubble effect&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      When viewed at the level of the URL (2C), the study is consistent with a negligible Filter Bubble effect
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h3 id=&#34;so-which-is-it&#34;&gt;So which is it?&lt;/h3&gt;
&lt;p&gt;First, what&amp;rsquo;s the difference between these two figures? 2B aggregates things at the domain level (e.g., &lt;a href=&#34;https://www.yahoo.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;www.yahoo.com&lt;/a&gt;) while 2C aggregates things at the URL level (e.g., &lt;a href=&#34;https://www.yahoo.com/news/pence-trumps-indictment-anyone-puts-002049678.html%29&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://www.yahoo.com/news/pence-trumps-indictment-anyone-puts-002049678.html)&lt;/a&gt;. That means 2B treats all shares from yahoo.com the same, while 2B looks at each story separately.&lt;/p&gt;
&lt;p&gt;If you&amp;rsquo;re like me, when you think of political news, you have in mind domains like FoxNews.com or MSNBC.com, where it&amp;rsquo;s likely that the website itself has a distinct partisan flavor.&lt;/p&gt;
&lt;p&gt;But YouTube.com and Twitter.com both appear in the &amp;ldquo;Top 100 Domains by Views&amp;rdquo; in the study&amp;rsquo;s SM, which obviously host a ton of both far left and far right or &amp;ldquo;mixed&amp;rdquo; content. And indeed, Figure S10 below shows that there are an array of domains that host some far right content and some far left content.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-some-domains-host-some-far-right-content-urls-and-some-far-left-content&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BailonFigS10.jpeg&#34; alt=&#34;Figure S10 from González-Bailón et al 2023: Some domains host some far right content and some far left content.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Some domains host some far right content (URLs) and some far left content.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;!-- There&#39;s also just something about content that makes an argument that we seem to want to share, as suggested by analysis in my [dissertation](https://www.dropbox.com/s/zfw1d9j60hqjil7/sudiss.pdf?raw=true), which shows that NYT editorials are more likely than content from other sections to be shared (via email). Consider two OpEds at the NYT---one by conservative columnist Bret Stevens, the other by well-known liberal Nicholas Kristof. The former is going to be shared more by conservatives, the latter more by liberals.  --&gt;
&lt;p&gt;But even if we&amp;rsquo;re talking about NYTimes.com it&amp;rsquo;s not hard to see that for example, conservatives might be more likely to share conservative Op Eds from Bret Stevens, while liberals may be more likely to share Op Eds from Nicholas Kristof.&lt;/p&gt;
&lt;p&gt;If you aggregate your analysis to the domain level, you&amp;rsquo;ll miss this aspect of media polarization.&lt;/p&gt;
&lt;h3 id=&#34;domains-or-urls&#34;&gt;Domains or URLs&lt;/h3&gt;
&lt;p&gt;So which is right? &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; suggests that we should prefer the URL-level analysis. Here&amp;rsquo;s the passage in the discussion section, which describes why analyzing media polarization at the level of the domain is problematic:&lt;/p&gt;
&lt;p&gt;&amp;ldquo;As a result of social curation, exposure to URLs is systematically more segregated than exposure to domains&amp;hellip; &lt;em&gt;A focus on domains rather than URLs will likely understate, perhaps substantially, the degree of segregation in news consumption online.&lt;/em&gt;&amp;rdquo; (Emphasis added)&lt;/p&gt;
&lt;p&gt;What&amp;rsquo;s more, past work (co-authored by one of the lead authors) shows that &lt;a href=&#34;https://osf.io/vbwer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;domain-level analysis can indeed mask “curation bubbles”&lt;/a&gt; in which &amp;ldquo;specific stories attract different partisan audiences than is typical for the outlets that produced them.&amp;rdquo;&lt;/p&gt;
&lt;p&gt;You can see this in the data clear as day&amp;mdash;let&amp;rsquo;s go back to Figure 2A, which shows a massive increase in estimated segregation when using URLs rather than domains:&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-figure-2a-shows-much-higher-levels-of-audience-segregation-when-you-look-at-the-url-level-rather-than-the-domain-level&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BailonFig2.jpg&#34; alt=&#34;Figure 2 from González-Bailón et al 2023: There are much higher levels of audience segregation when you look at the URL level rather than the domain level&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Figure 2A shows much higher levels of audience segregation when you look at the URL level rather than the domain level
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Returning to Figures 2B and 2C, it seems that the potential audience at domain level is &lt;em&gt;artificially less segregated&lt;/em&gt;, due to the aggregation at the level of the domain.&lt;/p&gt;
&lt;h3 id=&#34;what-explains-the-filter-bubble-discrepency&#34;&gt;What explains the &amp;lsquo;Filter Bubble discrepency&amp;rsquo;?&lt;/h3&gt;
&lt;p&gt;One possibility is that posts linking to content from &amp;ldquo;mixed&amp;rdquo; domains like YouTube, Reddit, Twitter, Yahoo, etc. do not score as well in feed-ranking. It&amp;rsquo;s possible that partisan content on these domains is more likely to be downranked as misinformation or spam, or maybe Facebook-native videos (which render faster/better) have an edge over YouTube, or perhaps there are domain-level features in feed ranking, or maybe there is some other reason that content from &amp;rsquo;non-mixed&amp;rsquo; domains just performs better in the rather complex recommendation system that powers Newsfeed ranking.&lt;/p&gt;
&lt;p&gt;Regardless, that would explain the results in Figure 2&amp;mdash;making the potential audience look artificially broader than the actual audience, when you analyze content at the domain level.&lt;/p&gt;
&lt;h3 id=&#34;what-about-the-reverse-chron-experiment&#34;&gt;What about the Reverse-Chron experiment?!&lt;/h3&gt;
&lt;p&gt;Surely we can paint a fuller picture of the impact of algorithmic ranking on media polarization with that other excellent recent &lt;em&gt;Science&lt;/em&gt; paper which looked at the &lt;em&gt;causal&lt;/em&gt; effect of turning off Newsfeed ranking. Maybe we can cross-reference that paper and get a clearer picture of what&amp;rsquo;s happening.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://www.science.org/doi/10.1126/science.abp9364&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Guess et al 2023&lt;/a&gt; shows that Newsfeed induces proportionally &lt;em&gt;more&lt;/em&gt; exposure cross-cutting sources but also more exposure to like-minded sources. It reduces exposure to moderate or mixed sources. Importantly, this is not just news that news sources link to, it&amp;rsquo;s all content that everyone posts, including life updates, pictures, videos, etc.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/GuessFig.jpg&#34; alt=&#34;Figure 2 from Guess et al 2023: Compared to a reverse chronologically-ranked feed, FB&amp;amp;rsquo;s ranking system induces a proportionally more exposure to &amp;amp;ldquo;like-minded&amp;amp;rdquo; sources but also more to cross-cutting sources, defined at the level of the entity posting.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Ok, but what about political content? In the supplimentary materials, we see that when it comes to &lt;em&gt;political content&lt;/em&gt;, Newsfeed ranking actually &lt;em&gt;decreases&lt;/em&gt; exposure to political content from like-minded sources.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/GuessChronScienceTabS20.jpg&#34; alt=&#34;Figure S20 from Guess et al 2023: A reverse chronologically-ranked Newsfeed induces a proportionally less exposure to &amp;amp;ldquo;like-minded&amp;amp;rdquo; sources but also less to cross-cutting sources, defined at the level of the entity posting.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;What about exposure to political content posted by cross-cutting sources? The SM doesn&amp;rsquo;t provide that, but it does provide a paragraph noting that Newsfeed &lt;em&gt;decreased&lt;/em&gt; exposure to political news from partisan sources relative to reverse-chron!&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/GuessChronScienceS3.3.jpg&#34; alt=&#34;S3.3 in Guess et al 2023: A reverse chronologically-ranked Newsfeed induces a proportionally less exposure to &amp;amp;ldquo;like-minded&amp;amp;rdquo; sources but also less to cross-cutting sources, defined at the level of the entity posting.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Now a big caveat here is that it&amp;rsquo;s clear from the main results that political content is not doing well in Newsfeed ranking. Note that there are &lt;a href=&#34;https://www.wsj.com/articles/facebook-politics-controls-zuckerberg-meta-11672929976&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;reports that the company decided to downrank political and news content in 2021&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Regardless, the results in the SM are not at all suggestive of a filter bubble, at least during the 2020 election&amp;mdash;the experimental results suggest that if anything feedranking is showing us &lt;em&gt;less&lt;/em&gt; polarizing content than we would see with reverse-chron.&lt;/p&gt;
&lt;h3 id=&#34;how-fb-groups-impact-estimates-of-algorithmic-segregation&#34;&gt;How FB Groups impact estimates of algorithmic segregation&lt;/h3&gt;
&lt;p&gt;I sent a much earlier draft of this to &lt;a href=&#34;https://www.asc.upenn.edu/people/faculty/sandra-gonzalez-bailon-phd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Sandra González-Bailón&lt;/a&gt; and &lt;a href=&#34;https://cssh.northeastern.edu/faculty/david-lazer/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;David Lazer&lt;/a&gt;, lead authors for &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt;. Sandra pointed me to Figure S14, which does show a slight increase in segregation post-ranking for URLs shared by users and pages, but shows the &lt;em&gt;opposite&lt;/em&gt; for content shared in those often-contentious Facebook groups.&lt;/p&gt;
&lt;!-- People may be especially likely to come across &#34;persons dissimilar to themselves, and with modes of thought and action unlike those with which they are familiar&#34; [(Mill cited in Mutz and Mondak, 2006)](https://www.polisci.upenn.edu/sites/default/files/mutz_mondak_2006.pdf). And it seems likely based on the plot below that group posts are treated differently from page posts and friend posts in Facebook feedranking.  --&gt;
&lt;p&gt;So it&amp;rsquo;s really &lt;em&gt;not&lt;/em&gt; that there&amp;rsquo;s no difference at all pre- and post- ranking, just that the difference is on average zero once you include groups. Of course, the population of people who see content from groups and/or pages in Newsfeed may be unusual, and future work should dig into this variation.&lt;/p&gt;
&lt;p&gt;I should also note that this small but real difference seems more or less consistent with what we &lt;a href=&#34;https://www.science.org/doi/10.1126/science.aaa1160&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;found in past work&lt;/a&gt;, which only examined news shared by users (excluding pages and groups).&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BailonFigS14.jpg&#34; alt=&#34;Figure S14 from González-Bailón et al 2023: There is a modest filter bubble effect among users and pages, and a &amp;amp;ldquo;reverse filter bubble&amp;amp;rdquo; for content shared in groups.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h3 id=&#34;the-algorithm-and-the-most-politically-engaged&#34;&gt;The algorithm and the most politically engaged&lt;/h3&gt;
&lt;p&gt;There is also a hint of an increase in segregation post-ranking among the most politically engaged 10% of Facebook users. However, the paper notes that this trend &amp;ldquo;is only clear for domain-level data,&amp;rdquo; which we&amp;rsquo;ve already established should not be used here. (Note that they define high political interest users as those in the &amp;ldquo;top 10% of engagement&amp;hellip; (comments, likes, reactions, reshares) with content classified as political on Facebook&amp;hellip;)&amp;rdquo;).&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BailonFigS19.jpg&#34; alt=&#34;Figure S19 shows a modest filter bubble effect among the most engaged users at the peak of the 2020 election.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;A similar pattern holds for the top 1% of users (Figure S23 in the SM).&lt;/p&gt;
&lt;h3 id=&#34;algorithmic-segregation-and-ideology&#34;&gt;Algorithmic segregation and ideology&lt;/h3&gt;
&lt;p&gt;Feed ranking seems to expose both conservatives and liberals to more liberal content. This is consistent with my priors that conservative content is more likely to violate policy and be taken down or subject to &amp;ldquo;soft actioning&amp;rdquo; (e.g., downranking) for borderline violations.&lt;/p&gt;
&lt;p&gt;So now things get messy&amp;mdash;should we really say liberals are in a filter bubble (and conservatives aren&amp;rsquo;t) if misinformation is included in that calculation?&lt;/p&gt;
&lt;!-- exposure to cross cutting content was thought to [reduce political participation](https://www.jstor.org/stable/3088437). --&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-feedranking-exposes-you-to-slightly-more-liberal-content&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BailonTabS8.jpeg&#34; alt=&#34;Feedranking exposes you to slightly more liberal content, presumably due to misinformation actioning.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Feedranking exposes you to slightly more liberal content.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;We can see a similar pattern when we look at exposure to cross-cutting content, which is the measure our 2015 Science paper used. Conservatives see more liberal content in feed than their friends share.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-replication-of-bakshy-et-al-2015&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BailonFigS11.jpg&#34; alt=&#34;Replication of Bakshy et al 2015, showing little effect of feed-ranking among content shared by friends. However, when including page and group content, feed-ranking plays a more important role&amp;amp;mdash;exposing liberals to proportionally less cross-cutting content and conservatives to comparatively more.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Replication of Bakshy et al 2015
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;We can also see that things have changed a lot since we wrote our original Science piece in 2015. Liberals seem to see &lt;em&gt;far&lt;/em&gt; more cross-cutting content, conservatives less.&lt;/p&gt;
&lt;!-- What about how this compares to others ways we get media on the internet? The evidence is not great here, but [Flaxman et al 2016](https://academic.oup.com/poq/article/80/S1/298/2223402) show that ideological segregation is higher for search than for social media websites, but lower for news aggregators and direct navigation to news websites.  --&gt;
&lt;h3 id=&#34;brevity-is-a-double-edged-sword&#34;&gt;Brevity is a double-edged sword&lt;/h3&gt;
&lt;p&gt;Science gives authors only limited space and it would have been difficult to dig into everything I&amp;rsquo;ve written about in 2 pages. I also know better than almost anyone just how much work went into these papers (I would bet thousands of hours for each of several authors), and how difficult it can be explain everything perfectly when you&amp;rsquo;re pulling off such a big lift. I should also point out that these papers are very nuanced, well-caveated, and careful not to overstate their results regarding the filter bubble or algorithmic polarization.&lt;/p&gt;
&lt;p&gt;Still, I do wish this work had squarely focused on URL-level analyses.&lt;/p&gt;
&lt;h3 id=&#34;what-this-means-for-other-studies&#34;&gt;What this means for other studies&lt;/h3&gt;
&lt;p&gt;This also means that past estimates of ideological segregation based domain-level analysis probably &lt;em&gt;understate&lt;/em&gt; media polarization in a big way. This includes those based on &lt;a href=&#34;https://journalqd.org/article/view/2586/2683&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Facebook data&lt;/a&gt;, &lt;a href=&#34;https://onlinelibrary.wiley.com/doi/epdf/10.1111/ajps.12589&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;browser data&lt;/a&gt;, on data describing various &lt;a href=&#34;https://academic.oup.com/poq/article/80/S1/298/2223402&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;platforms&lt;/a&gt;) or simply websites across the &lt;a href=&#34;https://web.stanford.edu/~gentzkow/research/echo_chambers.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;internet&lt;/a&gt;. I have said this for a long time but I did not think the magnitude was as strong as shown in &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;so-is-facebook-polarizing&#34;&gt;So, is Facebook polarizing?&lt;/h3&gt;
&lt;p&gt;It&amp;rsquo;s very hard to give a good answer to this question. In &amp;ldquo;The Paradox of Minimal Effects,&amp;rdquo; Stephen Ansolabehere points out that re-election depends overwhelmingly on whether the country is prosperous and at peace, not what happens with media politics. This is thought to be because people selectively consume media, which serves mainly to reinforce their beliefs; while at the same time, the sum total of people&amp;rsquo;s private lived experiences correponds reasonably well to aggregated economic data.&lt;/p&gt;
&lt;p&gt;There&amp;rsquo;s an argument that social media exists somewhere in between the conventional media and one&amp;rsquo;s lived experiences. And what about evidence? As Sean Westwood and I have shown, partisan selectivity is far less severe when you &lt;a href=&#34;https://journals.sagepub.com/doi/10.1177/0093650212466406&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;add a social element to news consumption&lt;/a&gt;. What&amp;rsquo;s more, field-experimental work I did shows that &lt;a href=&#34;https://www.dropbox.com/s/nu39148ukbab34r/CH7brief.pdf?raw=true&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;increasing the prominence of political news in Facebook&amp;rsquo;s Newsfeed&lt;/a&gt; shifted issue positions toward the majority of news encountered (left-leaning), particularly among political moderates.&lt;/p&gt;
&lt;p&gt;Tom Cunningham recently wrote a nice &lt;a href=&#34;https://tecunningham.github.io/posts/2023-07-27-meta-2020-elections-experiments.html#other-evidence-on-media-and-polarization&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;summary of some of the evidence related to the question of whether any kind of media might increase affective polarization&lt;/a&gt;, which we discussed at length.&lt;/p&gt;
&lt;p&gt;The evidence suggests any effect is likely small. First, we see that while social media is a global phenomenon, affective polarization is not&amp;mdash;the &lt;a href=&#34;https://direct.mit.edu/rest/article-abstract/doi/10.1162/rest_a_01160/109262/Cross-Country-Trends-in-Affective-Polarization?redirectedFrom=fulltext&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;UK, Japan, and Germany have seen affective depolarization&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Second, perhaps the highest quality experimental study on this question I&amp;rsquo;ve seen is &lt;a href=&#34;https://osf.io/jrw26/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Broockman and Kalla (2022)&lt;/a&gt;, which finds that paying heavy Fox News viewers to watch CNN has generally depolarizing effects, though as Tom points out, finds null effects on traditional measures of affective polarization.&lt;/p&gt;
&lt;p&gt;Third, Tom and I have also discussed an excellent experimental study attempting to shed light on this: &amp;ldquo;&lt;a href=&#34;https://www.aeaweb.org/articles?id=10.1257/aer.20190658&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Welfare Effects of Social Media&lt;/a&gt;,&amp;rdquo; which concludes that Facebook is likely polarizing. They find that &amp;ldquo;deactivating Facebook for the four weeks before the 2018 US midterm election&amp;hellip; makes people less informed, it also makes them less polarized by at least some measures, consistent with the concern that social media have played some role in the recent rise of polarization in the United States.&amp;rdquo;&lt;/p&gt;
&lt;p&gt;The study defines political polarization in an unusual way&amp;mdash;including congenial media exposure&amp;mdash;how much news you see from your own side&amp;mdash;in its polarization index. Most political scientists would consider congenial media exposure as &lt;em&gt;the thing that might cause polarization&lt;/em&gt;, but not an aspect of polarization in and of itself.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-allcott-et-al-2020-figure-3-demonstrates-the-biggest-effect-is-on-a-measure-of-media-exposure&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/AlcottetalFigure3.jpg&#34; alt=&#34;Allcott et al 2020 Figure 3&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Allcott et al 2020 Figure 3 demonstrates the biggest effect is on a measure of media exposure
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;They do explain that their effects on affective polarization are not significant and they don&amp;rsquo;t try to hide what&amp;rsquo;s going into the measure. But you have to read beyond the abstract and the media headlines to really understand this point.&lt;/p&gt;
&lt;!-- They make the following claim in a footnote: &#34;Online Appendix Table A16 shows that the effect on the political polarization index is *robust to excluding each of the seven individual component variables in turn*, although the point estimate moves toward zero and the unadjusted p-value rises to 0.09 when omitting congenial news exposure.&#34;  --&gt;
&lt;p&gt;Notably, in a robustness test in the appendix, the effect on the polarization loses statistical significance when you exclude this variable.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-allcott-et-al-2020-figure-a16-shows-that-the-effect-on-polarization-is-not-significant-at-p005-if-you-exclude-congenial-media-exposure&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/AlcottetalFigureA16.jpg&#34; alt=&#34;Allcott et al 2020 Figure A16&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Allcott et al 2020 Figure A16 shows that the effect on polarization is not significant at P&amp;lt;0.05 if you exclude congenial media exposure.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;That means the folks who didn&amp;rsquo;t deactivate Facebook had higher levels of political knowledge and higher levels of issue polarization. This makes sense because if a person doesn&amp;rsquo;t know where the parties stand on an issue, she is less likely to be sure about where she ought to stand.&lt;/p&gt;
&lt;h3 id=&#34;implications-for-how-this-was-publicized&#34;&gt;Implications for how this was publicized&lt;/h3&gt;
&lt;p&gt;All of this is relevant in light of the controversial Science Cover, which suggests Facebook&amp;rsquo;s &lt;em&gt;algorithms&lt;/em&gt; are &amp;ldquo;Wired to Split&amp;rdquo; us. It may be true, but the evidence across all 4 &lt;em&gt;Science&lt;/em&gt; and &lt;em&gt;Nature&lt;/em&gt; papers is not decisive on this question.&lt;/p&gt;
&lt;p&gt;None of the experiments published so far show an impact on affective or ideological polarization. What&amp;rsquo;s more, the proper URL-level analyses in &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; show only a modest &amp;lsquo;Filter Bubble&amp;rsquo; for certain subsets of the data, and when it comes to political news, and the reverse-chronological feed ranking experiment shows Newsfeed ranking feeds us &lt;em&gt;less&lt;/em&gt; polarized political content than would see in a reverse-chron Newsfeed.&lt;/p&gt;
&lt;p&gt;Of course, the spin from the Meta Comms team that these &lt;a href=&#34;https://www.wsj.com/articles/does-facebook-polarize-users-meta-disagrees-with-partners-over-research-conclusions-24fde67a&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;results are exculpretory&lt;/a&gt; is also highly problematic. This claim is not only wrong but amatuerish and self-defeating from a strategic perspective, and I was surprised to read about it.&lt;/p&gt;
&lt;p&gt;For all the amazing work done to produce the experimental results, the data are too noisy to detect small but potentially compounding effects on polarization as suggested in &lt;a href=&#34;https://statmodeling.stat.columbia.edu/author/dean/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;a post-publication review from Dean Eckles&lt;/a&gt; and &lt;a href=&#34;https://tecunningham.github.io/posts/2023-07-27-meta-2020-elections-experiments.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tom Cunningham&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;What&amp;rsquo;s more, even the excellent work done here in &lt;a href=&#34;https://www.science.org/doi/full/10.1126/science.ade7138&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;González-Bailón et al 2023&lt;/a&gt; does not speak to the question of effects on polarization that other key recommender systems at Facebook may have: the People You May Know (PYMK) algorithm, which facilitates network connections on the website, along with the Pages You Might Like (PYML) and Groups You Might Like (GYML). The authors make a similar point in the Supplementary Materials, S3.2&amp;mdash;pointing out that inventory, or the &amp;ldquo;potential audience,&amp;rdquo; &amp;ldquo;results from another curation process determining the structure and composition of the Facebook graph, which itself results from social and algorithmic dynamics.&amp;rdquo;&lt;/p&gt;
&lt;p&gt;This means that we should &lt;em&gt;not&lt;/em&gt; necessarily conclude that exposure is all about individual choices and not algorithms based on the sum total of evidence we have (a point I should have better emphasized &lt;a href=&#34;https://solomonmg.github.io/pdf/Science-2015-Bakshy-1130-2.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;in past work&lt;/a&gt;)&amp;mdash;algorithms may play an important role and as usual, more research is needed.&lt;/p&gt;
&lt;p&gt;It would seem to me that both Science and Meta Comms are both going beyond the data here.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-wired-to-split&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/ScienceCoverWiredtoSplit.jpeg&#34; alt=&#34;The Science Cover suggests Facebook&amp;amp;rsquo;s algorithms are &amp;amp;ldquo;Wired to Split&amp;amp;rdquo; us.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Wired to Split
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h3 id=&#34;disclosures&#34;&gt;Disclosures&lt;/h3&gt;
&lt;p&gt;As noted above, from 2018-2020, I was tech lead for Social Science One, which gave external researchers &lt;em&gt;direct&lt;/em&gt; access to data (the &lt;a href=&#34;https://solmessing.netlify.app/pdf/Facebook_DP_URLs_Dataset.pdf&#34;&gt;&amp;lsquo;Condor&amp;rsquo; URLs data set&lt;/a&gt;) via differential privacy. However, that project has not yielded much research output for a number of organizational and operational reasons, including the fact that differential privacy is not yet suitable for such a complex project.&lt;/p&gt;
&lt;p&gt;While at Facebook, I personally advocated (with Annie Franco) for the collaboration model used in the Election 2020 project, wherein external researchers would collaborate with Facebook researchers. That model would have to shield the research from any interference from Facebook&amp;rsquo;s Communications and Policy arm, which would violate scientific ethics. It would involve pre-registration, not merely to ensure scientific rigor but to protect against conflicts of interest and selective reporting of results. However, I left Facebook in January of 2020 and have not been deeply involved in the project since.&lt;/p&gt;
&lt;p&gt;I recently left Twitter (requesting to be in the first rounds of layoffs after Elon Musk took over) and started a job at NYU&amp;rsquo;s CSMaP lab when my employment with Twitter ended. There are authors who are affiliated with my lab on the paper, including one of the PIs, Josh Tucker. My graduate school Advisor, Shanto Iyengar is also on the paper, and I consider the majority of the authors to be my colleagues and friends.&lt;/p&gt;
&lt;p&gt;See also my &lt;a href=&#34;https://solmessing.netlify.app/disclosures/&#34;&gt;disclosures page&lt;/a&gt;.&lt;/p&gt;
&lt;!-- I do wish the samples and timeframes for the ranking experiments were bigger so we could understand potentially smaller effects, which may be very important.  --&gt;</description>
    </item>
    
    <item>
      <title>On BlueSky</title>
      <link>https://solmessing.netlify.app/post/bluesky-quasi-decentralized-social-network/</link>
      <pubDate>Mon, 03 Apr 2023 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/bluesky-quasi-decentralized-social-network/</guid>
      <description>&lt;p&gt;TL/DR Summary&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;BlueSky has a chance to dethrone twitter right now, but that path is narrow. &lt;/li&gt;
&lt;li&gt;Its exclusive invite only model means its user base is now small, elite, and homogenous with few bad actors. Almost everyone likes it. But the real test will be when it opens to the public. &lt;/li&gt;
&lt;li&gt;It is designed for true account portability and in theory should prevent a single company from owning the entire network as it scales up. &lt;/li&gt;
&lt;li&gt;However, it’s unclear if an ecosystem of small companies can do the job of content moderation in the same ways that centralized social networks do. The same is true of running modern feed-ranking and follow-recommendation systems. &lt;/li&gt;
&lt;li&gt;There will be growing pressure to make money using ads to cover costs as the network scales up, which will incentivize centralizing key data and resources, undermining the original model.&lt;/li&gt;
&lt;li&gt;Future possibilities include: (1) BlueSky remains de-facto centralized, &amp;ldquo;in beta&amp;rdquo; until it can get composable moderation right, which turns out to be the foreseeable future; (2) big players (Google, Facebook) join the party and dominate the ecosystem; (3) small, unmoderated, ad-free apps proliferate and the network becomes overrun with spam, NSFW, hate, scams and gifts that come with a lack of moderation.&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;p&gt;Pretty much everyone at Twitter&amp;mdash;and especially Jack Dorsey&amp;mdash;has long known that BlueSky could replace Twitter. When I joined Twitter in 2021, I soon learned our CEO was terribly unpopular internally, sporting a job approval rating under 40 percent, by far the lowest of any executive at the company.&lt;/p&gt;
&lt;p&gt;In fact, Jack was obsessed with decentralization, he seemed convinced that it was a mistake to have Twitter organized as a corporation, and he would rant about this on company-wide calls, which he seemed to be taking from caves in South Asia. This is when everyone else at the company was desperately trying to increase revenues to save the company from implosion.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-this-photo-of-jack-dorsey-captures-his-general-aspect-on-many-all-hands-calls&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/JackDorseyInCaveCreditTwitter.jpg&#34; alt=&#34;A photo of young Jack Dorsey in a cave.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      This photo of Jack Dorsey captures his general aspect on many all-hands calls.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Enter BlueSky, which would decentralize Twitter. Jack launched the initiative in 2019, and his plan was to migrate Twitter to this new protocol. It puts user data including posts and follow lists on open, public portable data servers (PDSs) that mean true account portability. Any business or organization could index the those servers, or what I will call the “BlueSkyVerse” (technically the &lt;a href=&#34;https://blueskyweb.xyz/blog/10-18-2022-the-at-protocol&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;AT Protocol&lt;/a&gt;), rank posts, and create a front end interface.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-the-bluesky-app-reads-posts-and-the-follow-graph-from-portable-data-servers-centralizing-them-in-an-index-ranking-and-dislaying-them-for-users&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/BlueSkyVerse-illustration.jpg&#34; alt=&#34;A node labelled as BlueSky or App sits atop various Portable Data Server (PDS) nodes, with arrows (edges) pointing to them. A caption to the right of the App node reads &amp;amp;ldquo;indexing, ranking, moderation, UX&amp;amp;rdquo; and a caption to the right of the PDS nodes reads &amp;amp;ldquo;Open AT protocol: user posts, likes, follow graph.&amp;amp;rdquo;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      The BlueSky App reads posts and the follow graph from Portable Data Servers, centralizing them in an index, ranking, and dislaying them for users.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;But wait a minute! Remember during Elon Musk’s acquisition how everyone said that the value of twitter isn’t the tech, but rather the network of creators and the communities that exist there? If you decouple that network from the platform you give up your most valuable asset—Google, Meta, others can index the network, develop a user interface, create some algorithms, show ads, and eat your lunch.&lt;/p&gt;
&lt;p&gt;And yet, Jack was about to do just that, filling Twitter’s moat by turning its most valuable asset into a protocol. Of course, this did not go over well with employees who weren&amp;rsquo;t independently wealthy, nor the board, who eventually pushed him out.&lt;/p&gt;
&lt;p&gt;BlueSky nicely captures the essence of Jack’s reign as half-time CEO: how little he cared about Twitter as a business and how much he cared about Twitter as an ecosystem.&lt;/p&gt;
&lt;p&gt;But back to the question everyone cares about right now: will this new system lead to a better social network, or set of networks? Is this finally the Twitter alternative we’re looking for?&lt;/p&gt;
&lt;p&gt;Make no mistake about it&amp;mdash;BlueSky was designed by Twitter to replace Twitter. This makes it very different from the other new social media protocols, apps, etc. that we’ve seen come on the scene of late. As &lt;a href=&#34;https://mastodon.social/@gruber/110314523447694321&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;John Gruber put it&lt;/a&gt;, “If you hated Twitter, you’ll like Mastodon. If you liked Twitter, you’ll love BlueSky.”&lt;/p&gt;
&lt;p&gt;So it’s a contender, despite how hard it is to start a social network from scratch. And don’t get any funny ideas about a post-surveillance-capitalism social network&amp;mdash;if BlueSky takes off, it will most likely devolve into a less-moderated, less-profitable version of Twitter, Inc (aka Twitter 1.0). It will indeed encourage competition for front-end interfaces to explore the BlueSkyVerse. But the biggest challenges that social networks have to face—content moderation, discoverability, and monetization—require big technical and infrastructural investments to do well. They may only be viable for well-capitalized companies that generate big profits.&lt;/p&gt;
&lt;p&gt;But of course, I would be very nervous if I still worked at Twitter.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Will it work?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Now is a unique opportunity for a Twitter rival. Twitter CEO Elon Musk tends to say &lt;a href=&#34;https://www.cnn.com/2022/10/30/business/musk-tweet-pelosi-conspiracy/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;all manner of nutty things&lt;/a&gt;, he has &lt;a href=&#34;https://techcrunch.com/2022/11/21/elon-musk-twitter-netzdg-test/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;decimated Twitter&amp;rsquo;s trust and safety org&lt;/a&gt;, and cut staffing by more than 80%. And the company slashed infrastructure budgets needed for automated content moderation&amp;mdash;internal sources say the company has cut 3bn since peak spending prior to the recession, while external accounts say &lt;a href=&#34;https://www.reuters.com/technology/musk-orders-twitter-cut-infrastructure-costs-by-1-bln-sources-2022-11-03/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Musk ordered a 1bn cut himself&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;It shows: in the wake of the Allen massacre on Saturday, &lt;a href=&#34;https://www.dallasnews.com/news/2023/05/11/gore-conspiracies-spread-on-elon-musks-loosely-moderated-twitter-after-allen-shooting/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;graphic videos and misinformation spread across the platform&lt;/a&gt;. Advertisers don&amp;rsquo;t want to risk putting their brands next to that kind of content and &lt;a href=&#34;https://www.cnbc.com/2022/11/01/ad-giant-ipg-advises-brands-to-pause-twitter-spending.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;many have suspended advertising&lt;/a&gt; on the platform.&lt;/p&gt;
&lt;p&gt;We’ve all wondered which alternative social media system might replace Twitter. Could it be Mastodon, Spoutible, Post News, maybe Substack Notes? Or perhaps Truth Social or Gab or Gettr?!&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-the-only-ads-i-saw-on-gabcom-were-ads-for-advertising-on-gabcom&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/NoAdsonGab.jpg&#34; alt=&#34;A screenshot from Gab.com, with a post showing a flag of the UN with the text: &amp;amp;ldquo;Need to burn a flag? Make it this one.&amp;amp;rdquo;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      The only ads I saw on Gab.com were ads for advertising on Gab.com.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;I’m guessing it’s not going to be those other networks. The new centralized social network entrants—Spoutible, Post News, and Substack Notes—feel sterile and inauthentic when you first get started, partially because they are built around conventional media outlets, partially because they didn’t pay enough attention to discoverability in onboarding. Gettr/gab/truth social have libertarian-borderline-right-wing moderation setups, and the vast majority of people on Twitter have little interest in a right-wing echo chamber where there&amp;rsquo;s no one to troll.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-you-can-get-the-best-designers-and-engineers-on-the-planet-but-if-you-show-people-a-blank-timeline-and-recommendations-to-follow-a-bunch-of-people-theyve-never-heard-of-no-one-is-going-to-use-your-platform&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/SpoutableBlankTimeline.jpg&#34; alt=&#34;A screenshot from Spoutible showing a blank timeline.&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      You can get the best designers and engineers on the planet but if you show people a blank timeline and recommendations to follow a bunch of people they&amp;rsquo;ve never heard of, no one is going to use your platform.
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Mastodon is losing steam for many reasons&amp;mdash;onboarding is terribly confusing, it’s broken into communal servers that are all very different but that all seem uptight. Moderation there has been characterized as “&lt;a href=&#34;https://mastodon.social/@gruber/110328355532624579&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;petulant nannyism&lt;/a&gt;.”&lt;/p&gt;
&lt;p&gt;Like Twitter, and &lt;a href=&#34;https://werd.io/2023/the-fediverse-and-the-at-protocol&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;unlike Mastodon&lt;/a&gt;, BlueSky can surface content from this entire web of activity across the BlueSkyVerse and delight you with memes and witticisms, many of which were about &lt;a href=&#34;https://faineg.substack.com/p/how-i-accidentally-ruined-bluesky&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;”Sexy” ALF (yes, the 80s TV star)&lt;/a&gt; when I signed up.&lt;/p&gt;
&lt;p&gt;Many beta users say BlueSky feels like a breath of fresh air, like a throwback to early Twitter. For now, BlueSky is invite-only and so missing are the scammers, crypto bros, right-wing nuts, and tone policing randos looking for followers you find on Twitter. It feels more communal and less exhausting. Unclear how long that will last.&lt;/p&gt;
&lt;p&gt;So maybe BlueSky has a legit claim to the Throne of Discourse, post-Twitter.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Content Moderation&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;First of all, content moderation is not just a “nice-to-have” thing that keeps the press happy. Facebook and others have found that content moderation &lt;a href=&#34;http://tecunningham.github.io/2023-04-28-ranking-by-engagement.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;increases retention&lt;/a&gt;. And look at the flip side: most people don’t want to hang out at what Mike Masnick calls “&lt;a href=&#34;https://www.techdirt.com/2023/05/04/on-social-media-nazi-bars-tradeoffs-and-the-impossibility-of-content-moderation-at-scale/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Nazi bars&lt;/a&gt;,” which is what platforms with permissive moderation policies will often become known for, whether they are actually Nazis or just radical free-speech advocates. Once that happens, kiss a lot of your core user base and ad revenue goodbye&amp;mdash;which is what seems to be happening at Twitter.&lt;/p&gt;
&lt;p&gt;Of course, content moderation is the bane of the modern social media network. It’s expensive, &lt;a href=&#34;https://www.techdirt.com/2019/11/20/masnicks-impossibility-theorem-content-moderation-scale-is-impossible-to-do-well/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;it will always be wrong&lt;/a&gt;, it can easily create a PR dumpster file, and its benefits are extremely difficult to measure. This new protocol was designed with content moderation in mind so let me break that down before talking about the problems that will surely come up.&lt;/p&gt;
&lt;p&gt;On BlueSky, speech happens on your PDS, but reach happens on the centralized app—Bluesky for now. And they &lt;a href=&#34;https://blueskyweb.xyz/blog/4-13-2023-moderation&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;are in fact moderating&lt;/a&gt;, so if they find a post that violates policy, they may take it off their app. It’s still up on the PDSs, it’s just not indexed in BlueSky. So great, it allows for a slightly truer form of “freedom of speech but not reach.”&lt;/p&gt;
&lt;p&gt;How does this actually work? The BlueSky team wants to create a “&lt;a href=&#34;https://blueskyweb.xyz/blog/4-13-2023-moderation&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;moderation ecosystem&lt;/a&gt;,” in which labels (“spam”, “nsfw”) can be created by anyone, and apps like BlueSky can then choose what labels to act upon. Right now, it’s completely centralized at BlueSky, and they have an automated layer and decisions are made by “server administrators.” Eventually though, there will be other label sources, other apps besides BlueSky and many servers beyond bsky.social.  They’re proposing a “choose your own moderation” approach.&lt;/p&gt;
&lt;p&gt;OK what are the downsides?&lt;/p&gt;
&lt;p&gt;First, there are key parts of moderation that raise questions under this framework. If you doxx someone’s home address for targeted harassment, post a bunch of Child Sexual Abuse Material (CSAM) or non-consensual sexual imagery, it feels insufficient to merely de-index those posts. There are cases where it &lt;a href=&#34;https://www.nytimes.com/2023/05/03/technology/dorsey-musk-twitter-bluesky-nostr.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;may not be legally sufficient&lt;/a&gt; under the Digital Services Act, NetzDG, or U.S. Copyright Law.&lt;/p&gt;
&lt;p&gt;The spam-detection arms race is another example—the more you are open with how it works, the faster the spammers get around your detection systems. Somewhat relatedly, the fact that blocklists are public on BlueSky due to the BlueSkyVerse architecture, is &lt;a href=&#34;https://twitter.com/MattBinder/status/1652142389165797377?s=20&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;already stirring controversy&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Finally, a big part of a healthy information ecosystem is keeping bad actors off your platform in the first place. In centralized networks, that’s often done by IP screening, cell phone/text message screening, email validation, and/or by using other private data. But a PDS hosts public data, so the centralized app would need to create parallel user accounts to collect and maintain that data.&lt;/p&gt;
&lt;p&gt;All that means it’s difficult to see an alternative to a world where BlueSky and other AT apps need to start collecting private user data, even if it’s inconsistent with the clean decentralized, portable data model illustrated above. The line between PDS and user account will get very fuzzy very fast.&lt;/p&gt;
&lt;p&gt;And, once apps do this for content moderation, wouldn’t they also wish to do it for advertising as well? Content moderation isn’t free.&lt;/p&gt;
&lt;p&gt;Right now, signups are based on invites, which helps keep out bad actors. But eventually BlueSky will need to open up fully once it’s out of beta.&lt;/p&gt;
&lt;p&gt;When that happens, the job of content moderation will be far more complex than in a place like Mastodon, because the BlueSky architecture is meant to enable “scale and global discoverability.”  With Mastodon/the Fediverse, each server has its own policies, norms, and content moderation, which is far simpler in its small, federated worlds. In the BlueSkyVerse, you have no choice but to scale up moderation.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Recommender systems in the BlueSkyVerse&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Will BlueSky be incentivized to build a feed-ranking system into their product and start logging the vast scope of data that inspired the phrase “surveillance capitalism?” They have already started down that path—in fact they’ve built the BlueSkyVerse to facilitate global discovery—large scale indexing and ranking across all PDSs in the network.&lt;/p&gt;
&lt;p&gt;Right now, the “What’s Hot” feed does global discovery, but in a way that is pretty basic—it’s showing popular stuff from the last 30 minutes. For now this is fine, &lt;a href=&#34;http://tecunningham.github.io/2023-04-28-ranking-by-engagement.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;it’s the core of most modern recommender systems&lt;/a&gt; in social media websites.&lt;/p&gt;
&lt;p&gt;Contrast this with Mastodon, where you can technically follow people from another server but the system isn’t designed so servers index each other and form one network. This is an important reason I think BlueSky could have legs, but Mastodon will probably not replace Twitter.&lt;/p&gt;
&lt;p&gt;Setting aside any monetary pressures facing BlueSky for a minute, I suspect they will be driven toward increased data collection and deployment, simply because you need to do that to move the metrics that tell you your product is improving. This may be further cemented by the culture of modern engineering organizations—where engineering leaders and PMs ruthlessly focus on moving a “north star” metric, which is almost alway some variant of time spent. “Time spent, daily active users, session counts, these are measures of whether you’re making your product better—the fact that they are all highly correlated with potential ad revenue is coincidental.&lt;/p&gt;
&lt;p&gt;Of course, to do anything like what Twitter and Facebook do with their recommender systems—for both follow recommendations and for feed-ranking—will require a lot more resources. For the follow graph, that entails predicting which users are likely to form mutual follow relationships or satisfactory follow-only relationships, which can be done with shortcuts but is ultimately a difficult (graph machine learning) problem. For feed-ranking, that requires predicting what users are likely to interact with what content, which both Twitter and Facebook had entire divisions of engineers and data scientists working on.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Pressures to centralize and monetize the BlueSkyVerse&lt;/strong&gt;
Venture capitalists and startups in Silicon Valley are always talking about “moats.” If you invest a great deal of resources to build a technology or a new marketplace, what’s to stop a competitor from drinking your milkshake?&lt;/p&gt;
&lt;p&gt;There’s an influential idea among “Web 3.0” circles, which is that Facebook, Instagram, and Twitter are the landlords of castles you can’t leave. That’s not supposed to happen this time—the BlueSkyVerse was designed around account portability and front-end/algorithmic competition. The hope is this will create an ecosystem of small companies doing bits and pieces of what big social media companies do today.&lt;/p&gt;
&lt;p&gt;At the same time, everything I’ve seen so far suggests that large investments are going be required to even start playing in the BlueSkyVerse—there are barriers to entry on data processing to even index it as users grow, to create a legit feed and UX, and to do content moderation at that kind of scale. Jack has given billions to the BlueSky team to get the system to where it is today.&lt;/p&gt;
&lt;p&gt;So what happens if the BlueSkyVerse really takes off? We might indeed see real competition for front-end apps that do custom algorithmic ranking and figure out innovative ways to moderate content. We might see further media fragmentation—perhaps front-end providers will try to differentiate themselves by topic or political orientation like television channels do.&lt;/p&gt;
&lt;p&gt;But running a modern social media website is expensive. If it grows as big as Twitter, indexing the BlueSkyVerse will become a challenge, same for running modern recommender systems. And if you want ad revenue you need content moderation, which you can’t solve with AI alone—you need humans in the loop, which means you don’t get the kind of economies of scale you’d see with automated systems. What’s more, you often need sensitive user data to do these things well, and you need bespoke solutions to new adversarial tactics you find. So it’s hard to fully rely on an external company for these solutions, as the creators of BlueSky seemed to envision.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The future of the network&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;I see a few possibilities if BlueSky gets really big: the first is that BlueSky the app simply dominates this system—they moved first, they understand the system, they can do content moderation, they figure out how to scale up, and they may decide to sell ads. At the same time if BlueSky does become “Twitter 3.0,” there have to be consequences to the fact that I can simply take my posts and follow-graph to a competing service and still be on the same network.&lt;/p&gt;
&lt;p&gt;Or maybe not. Maybe they will realize that the challenges of content moderation favor keeping the network as is, and the BlueSkyVerse will remain closed for a long time. Perhaps forever.&lt;/p&gt;
&lt;p&gt;But if it does really launch and open up, it seems likely that established tech starts to play—Google jumps in, dedicates a small fraction of the resources it used to fund Google+, indexes the BlueSkyVerse in a day, and boom… has a competitor to Facebook. Maybe Facebook jumps in too, but that’s a tricky proposition because once part of Facebook/Instagram has true account portability what happens to the rest of the company?&lt;/p&gt;
&lt;p&gt;Of course, another outcome that seems likely is a conservative social media front-end provider. Maybe Truth Social integrates with the BlueSkyVerse. It won’t make much money because many in that demographic seem happy with Twitter for now, and there will be substantial brand risk for potential advertisers.&lt;/p&gt;
&lt;p&gt;Finally, we might see pure anarchy. In this &amp;ldquo;race to the bottom&amp;rdquo; scenario, a set of small, unmoderated, ad-free apps proliferate. Since people don&amp;rsquo;t like ads, they use these apps. The network becomes overrun with spam, NSFW, hate, scams and gifts that come with a lack of moderation. Of course, it&amp;rsquo;s unclear these apps would be tolerated by the app stores, but this is one direction things might generally go.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>What can we learn from &#39;The Algorithm,&#39; Twitter&#39;s partial open-sourcing of it&#39;s feed-ranking recommendation system?</title>
      <link>https://solmessing.netlify.app/post/twitter-the-algorithm/</link>
      <pubDate>Mon, 03 Apr 2023 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/twitter-the-algorithm/</guid>
      <description>&lt;p&gt;Last Friday (2023-03-31) Twitter released what it calls “the algorithm,” which appears to be a highly redacted, incomplete part of code that governs the “for you” home timeline ranking system. And I saw nothing to suggest the parts of the code they put in the GitHub repository wasn’t authentic.&lt;/p&gt;
&lt;p&gt;It’s highly unusual for a tech company to open up a product at the core of its monetization strategy. The thinking is that the more engaging the content you show people right when they log in, the more likely they are to stick around. And the more you keep people logged in, the more they see ads. And the more data you can get to show them better ads!&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Transparency, or a distraction from closing the API?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Is this a step forward for transparency as Musk and Twitter would claim? I am skeptical. You can’t learn much from this release in and of itself&amp;mdash;you need the underlying model features, parameters, and data to really understand the algorithm. Those combine into a system that&amp;rsquo;s effectively different for everyone! So even if you had all that, you&amp;rsquo;d likely need to algorithmically audit the system to really get a handle on it.&lt;/p&gt;
&lt;p&gt;And Twitter made it &lt;a href=&#34;https://www.wired.com/story/twitter-data-api-prices-out-nearly-everyone/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;prohibitively expensive&lt;/a&gt; for external researchers to get that data through its API with the recent price updates ($500k/yr). So at the same time twitter is releasing this code, it’s made it incredibly difficult for research to &lt;em&gt;audit&lt;/em&gt; this code&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What&amp;rsquo;s in the code? Gossip and Rumors&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Ukraine&lt;/strong&gt; There were some &lt;a href=&#34;https://twitter.com/SolomonMg/status/1642845123531751425?s=20&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;initial reports&lt;/a&gt; that Twitter was downranking tweets about Ukraine. I looked at the code and can tell you those claims are wrong&amp;mdash;twitter has an audio-only &lt;a href=&#34;https://www.clubhouse.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Clubhouse&lt;/a&gt; clone called Spaces and that code is for that product, not ordinary tweets on hometimeline. What&amp;rsquo;s more, this is likely a label related only to &lt;strong&gt;crisis misinformation&lt;/strong&gt;, as per Twitter&amp;rsquo;s &lt;a href=&#34;https://help.twitter.com/en/rules-and-policies/crisis-misinformation&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Crisis Misinformation Policy&lt;/a&gt;.&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Uh, this code looks like it *only* relates to Twitter&amp;#39;s Spaces product. If so it is *not* used for down-ranking ordinary tweets in home timeline ranking. &lt;br&gt;&lt;br&gt;It&amp;#39;s called here with the other twitter Spaces spaces visibility models and labels: &lt;a href=&#34;https://t.co/7n8Tl7WVlJ&#34;&gt;https://t.co/7n8Tl7WVlJ&lt;/a&gt; &lt;a href=&#34;https://t.co/stSThUTvUe&#34;&gt;https://t.co/stSThUTvUe&lt;/a&gt;&lt;/p&gt;&amp;mdash; Sol Messing (@SolomonMg) &lt;a href=&#34;https://twitter.com/SolomonMg/status/1642560420392103936?ref_src=twsrc%5Etfw&#34;&gt;April 2, 2023&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;&lt;strong&gt;Musk Metrics&lt;/strong&gt; One of the most interesting things we learned from the code is that Twitter created an entire suite of metrics about Elon Musk’s personal twitter experience. The code shows they fed those metrics to the experimentation platform (Duck Duck Goose, or DDG), which at least historically has been used to evaluate whether or not to ship products.&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Twitter’s algorithm specifically labels whether the Tweet author is Elon Musk&lt;br&gt;&lt;br&gt;“author_is_elon”&lt;br&gt;&lt;br&gt;besides the Democrat, Republican and “Power User” labels&lt;a href=&#34;https://t.co/fhpBjdfifX&#34;&gt;https://t.co/fhpBjdfifX&lt;/a&gt; &lt;a href=&#34;https://t.co/orCPvfMTb9&#34;&gt;pic.twitter.com/orCPvfMTb9&lt;/a&gt;&lt;/p&gt;&amp;mdash; Jane Manchun Wong (@wongmjane) &lt;a href=&#34;https://twitter.com/wongmjane/status/1641884551189512192?ref_src=twsrc%5Etfw&#34;&gt;March 31, 2023&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;This episode is consistent with reporting that engineers are very concerned about how any features they ship &lt;a href=&#34;https://www.theverge.com/2023/2/9/23593099/elon-musk-twitter-fires-engineer-declining-reach-ftc-concerns&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;affect the CEOs personal experience on Twitter&lt;/a&gt;. And other &lt;a href=&#34;https://arstechnica.com/tech-policy/2023/02/report-musk-had-twitter-engineers-boost-his-tweets-after-biden-got-more-views/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;reporting has suggested that there may have been a Musk centric boost feature&lt;/a&gt; that shipped, and you would want exactly this kind of instrumentation to understand how that worked in practice.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Republican, Democrat Metrics&lt;/strong&gt; We also learned that Twitter is logging similar metrics for lists of prominent Democrat and Republican accounts, &lt;a href=&#34;https://www.yahoo.com/entertainment/twitters-recommendation-algorithm-is-now-on-github-200511112.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ostensibly to understand&lt;/a&gt; whether any features that they ship affect those sets of accounts equally. Now we know that &lt;a href=&#34;https://www.nature.com/articles/s41467-022-34769-6&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;conservative accounts tend to share more misinformation than liberal accounts on both Twitter&lt;/a&gt; and &lt;a href=&#34;https://www.science.org/doi/full/10.1126/sciadv.aau4586&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;on Facebook&lt;/a&gt;. And, &lt;a href=&#34;https://www.washingtonpost.com/technology/2023/02/08/house-republicans-twitter-files-collusion/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Musk has alleged that Democrats and Big Tech are colluding&lt;/a&gt; to enforce policy violation unequally across parties.&lt;/p&gt;
&lt;p&gt;But if you have these ``partisan equality&amp;rsquo;&amp;rsquo; stats as part of your ship criteria, perhaps on equal footing with policy violation frequency, you can see how &lt;strong&gt;this could really affect the types of health and safety features that actually make it to the site in production&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;This code was then comically removed via pull requests from Twitter. Because once you delete something on GitHub, it just goes away. Right?&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;which specific user groups, you might wonder? &lt;a href=&#34;https://t.co/wYSsUKm1pA&#34;&gt;pic.twitter.com/wYSsUKm1pA&lt;/a&gt;&lt;/p&gt;&amp;mdash; Colin Fraser (@colin_fraser) &lt;a href=&#34;https://twitter.com/colin_fraser/status/1641960748233662464?ref_src=twsrc%5Etfw&#34;&gt;April 1, 2023&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;&lt;strong&gt;Twitter Blue Boost&lt;/strong&gt; What’s more, we sorta knew that Twitter Blue users get a boost in feed ranking, but the code make it clear that it could double your score among people who don&amp;rsquo;t follow you, and quadruple it for those who do.&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Not sure what I expected, but interesting that first pull request on newly open-sourced &lt;a href=&#34;https://twitter.com/twitter?ref_src=twsrc%5Etfw&#34;&gt;@Twitter&lt;/a&gt; algo (&lt;a href=&#34;https://t.co/kAVP0zdzki&#34;&gt;https://t.co/kAVP0zdzki&lt;/a&gt;) is to downweight verified user multipliers. &lt;a href=&#34;https://t.co/PfqIdVTDnk&#34;&gt;pic.twitter.com/PfqIdVTDnk&lt;/a&gt;&lt;/p&gt;&amp;mdash; Caitlin Hudon (@beeonaposy) &lt;a href=&#34;https://twitter.com/beeonaposy/status/1641878347557883910?ref_src=twsrc%5Etfw&#34;&gt;March 31, 2023&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;As &lt;a href=&#34;https://twitter.com/jonathanstray/status/1642200687101501441&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Jonathan Stray pointed out&lt;/a&gt;, if this counts as a paid promotion, the FTC might require Twitter to label your tweets as ads. Now we kind of already knew this from Musks Twitter Blue announcement, but having evidence in the code might cross a different line for the FTC.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;So what about the ackshual algorithm? What does this say about feed ranking?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The code itself is there but it’s missing specifics&amp;mdash;key parameters, feature sets, and model weights are absent or abstracted. And obviously the data.&lt;/p&gt;
&lt;p&gt;The most critical thing we learned about Twitter’s ranking algorithm is probably from a readme file that former Facebook Data Scientist &lt;a href=&#34;https://twitter.com/jeff4llen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Jeff Allen&lt;/a&gt; found. If we take that at face value, a fav (twitter like) is worth half a retweet. A reply is worth 27 retweets, and a reply with a response from a tweets author is worth a whopping 75 retweets!&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;According to the Heavy Ranker readme, it looks like this is the &amp;quot;For you&amp;quot; feed ranking formula is&lt;br&gt;&lt;br&gt;Each &amp;quot;is_X&amp;quot; is a predicted probability the user will take that action on the Tweet.&lt;br&gt;&lt;br&gt;Replies are the most important signal. Very similar to MSI for FB.&lt;a href=&#34;https://t.co/Bmv7qg4voc&#34;&gt;https://t.co/Bmv7qg4voc&lt;/a&gt; &lt;a href=&#34;https://t.co/lWfaUboT6q&#34;&gt;pic.twitter.com/lWfaUboT6q&lt;/a&gt;&lt;/p&gt;&amp;mdash; Jeff Allen (@jeff4llen) &lt;a href=&#34;https://twitter.com/jeff4llen/status/1641901988047626241?ref_src=twsrc%5Etfw&#34;&gt;March 31, 2023&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;Now it’s not quite that simple&amp;mdash;what about when a tweet is first posted and there’s no data? Twitter’s deep learning system (in the heavy ranker) will do some heavy lifting and predict the likelihood of each of these actions based on the tweet author, their network, any initial engagements, the tweet text, and thousands of signals and embeddings.&lt;/p&gt;
&lt;p&gt;Of course, what happens in the first few minutes when a tweet is posted deeply shapes who sees and engages with it downstream in the future.&lt;/p&gt;
&lt;p&gt;[And the way this is implemented in practice is that the model handles all cases, but as you get more and more real time data on a tweet, those real time features dominate everything else and push those probabilities close to 1, see &lt;a href=&#34;https://twitter.com/SolomonMg/status/1642154005588504577?s=20&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;discussion here&lt;/a&gt;.]&lt;/p&gt;
&lt;p&gt;Now I should point out that there are some spammy accounts claiming to have found ranking parameters in the code. They’re wrong, those are used to &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/src/java/com/twitter/search/README.md&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;retrieve tweets from your network for candidate generation only&lt;/a&gt;. &lt;a href=&#34;https://lucene.apache.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Lucene&lt;/a&gt; is an open source search tool.&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Uh, this is for search query results, not home timeline ranking. &lt;br&gt;&lt;br&gt;See the thing that  says &amp;quot;luceneScoreParams&amp;quot;??&lt;br&gt;&lt;br&gt;Lucene is a search library: &lt;a href=&#34;https://t.co/993crKVHJi&#34;&gt;https://t.co/993crKVHJi&lt;/a&gt; &lt;a href=&#34;https://t.co/AdGtjCbOlx&#34;&gt;https://t.co/AdGtjCbOlx&lt;/a&gt;&lt;/p&gt;&amp;mdash; Sol Messing (@SolomonMg) &lt;a href=&#34;https://twitter.com/SolomonMg/status/1642563414970060800?ref_src=twsrc%5Etfw&#34;&gt;April 2, 2023&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;I should point out however, that some of the ``Earlybird&amp;rsquo;&amp;rsquo; code was at one point used in timeline ranking, and it appears that &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/EarlybirdTensorflowBasedSimilarityEngine.scala&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;it may be used in cr-mixer&lt;/a&gt;, which is used in candidate generation for &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/cr-mixer/README.md&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;out-of-network tweets&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Interestingly, &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/main/home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/filter/OutOfNetworkCompetitorURLFilter.scala&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Twitter appears to remove competitor URLs&lt;/a&gt;, perhaps only for tweets that are outside of network (you don&amp;rsquo;t follow the author).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What else goes into the ``the Algorithm?&amp;rsquo;&amp;rsquo;&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;What gets ranked in the first place? The other piece here is the ``TikTok&amp;rsquo;&amp;rsquo; part of the ranking algorithm, which is also incomplete without the models/data/parameters/etc. What I mean is the code that takes content from across the platform and says “I’m going to put this into your queue for the heavy ranker to sort out.”&lt;/p&gt;
&lt;p&gt;Now on Twitter often that historically meant tweets posted by or replied to by accounts you follow. But, Twitter realized it could find a lot more content for that heavy ranker magic.&lt;/p&gt;
&lt;p&gt;There’s a complex system that inserts tweets into your queue for ranking. This is called &lt;strong&gt;candidate generation&lt;/strong&gt; in the “recommendation system” subfield of applied computing.&lt;/p&gt;
&lt;p&gt;If you follow a lot of people on twitter like me, about &lt;strong&gt;half&lt;/strong&gt; of the candidate tweets in twitter’s ranked “for you” timeline at any given time are from people you follow.&lt;/p&gt;
&lt;p&gt;Now, if you don’t follow a ton of people, or if you have a new account, you can run out of these tweets, and then Twitter will try to find additional candidates so that you have ranked content. If so, means that this system is going to govern what in your home timeline feed like TikTok&amp;mdash;gathering content it predicts you’ll like from across the platform.&lt;/p&gt;
&lt;p&gt;This takes place in &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/cr-mixer/README.md&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;cr-mixer&lt;/a&gt;, and although some of the &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/cr-mixer/server/src/main/scala/com/twitter/cr_mixer/candidate_generation/CandidateSourcesRouter.scala&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;high level function calls are there&lt;/a&gt;, much of the code and the models appear to be missing, and many files come with this warning at the top: ``This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.&#39;&#39;&lt;/p&gt;
&lt;p&gt;Twitter seems to have made some of the systems public underlying candidate generation public, including its &lt;a href=&#34;https://github.com/twitter/the-algorithm/tree/7f90d0ca342b928b479b512ec51ac2c3821f5922/src/scala/com/twitter/simclusters_v2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;SimCluster model&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;BTW, I’d like to give a shout out to &lt;a href=&#34;https://twitter.com/vboykis&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Vicki Boykis&lt;/a&gt;, and &lt;a href=&#34;https://twitter.com/igorbrigadir&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Igor Brigadir&lt;/a&gt; who are &lt;a href=&#34;https://github.com/igorbrigadir/awesome-twitter-algo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;doing amazing work to map out the codebase&lt;/a&gt; and unearth exactly what’s missing and what’s not.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Trust and Safety&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;A lot of the code related to Trust and Safety is missing, presumably to prevent bad actors from learning too much and gaming those systems. However, there do seem to be some specifics about the kinds of things twitter considers borderline or violating that I don’t think were previously public.There are a bunch of safety parameters in the code, some of which are in Twitter’s policy documents, but some are not.&lt;/p&gt;
&lt;p&gt;There are entries like “HighCryptospamScore” that &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/visibilitylib/src/main/scala/com/twitter/visibility/rules/DownrankingRules.scala&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;appear in the code&lt;/a&gt;, which may give scammers hints about how to craft tweets to get around detection systems.  The same is true for &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/visibilitylib/src/main/scala/com/twitter/visibility/models/TweetSafetyLabel.scala#L115&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;code that contains links&lt;/a&gt; to “UntrustedUrl,” “TweetContainsHatefulConductSlur” for low, medium and high severity.&lt;/p&gt;
&lt;p&gt;There’s also a reference to a “Do Not Amplify” &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/visibilitylib/src/main/scala/com/twitter/visibility/models/SpaceSafetyLabelType.scala#L26&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parameter in the code&lt;/a&gt;, which was discussed in the twitter files but seems not to be publicly documented in it’s policies. There are entries like “AgathaSpam,” which refers to a propriety embedding used across the codebase. Twitter also has a bunch of visibility rules hardcoded in Scala that might be useful to bad actors trying to game the system, outlining what rules are in play for all tweets, new users, user mentions, liked tweets, realtime spam detection, etc. Finally, some of the consequences for those violations are &lt;a href=&#34;https://github.com/twitter/the-algorithm/blob/7f90d0ca342b928b479b512ec51ac2c3821f5922/visibilitylib/src/main/scala/com/twitter/visibility/rules/Action.scala&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;spelled out in Scala&lt;/a&gt; as well.&lt;/p&gt;
&lt;p&gt;Of course, it&amp;rsquo;s really hard to know with certainty that any of this wasn&amp;rsquo;t in public somehow before this release.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>A 2 million-person, campaign-wide field experiment shows how digital advertising affects voter turnout</title>
      <link>https://solmessing.netlify.app/publication/aggarwal20232/</link>
      <pubDate>Sun, 01 Jan 2023 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/aggarwal20232/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;Supplimentary materials at end of manuscript&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YMKVA1&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Replication materials&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Media coverage: &lt;a href=&#34;https://www.nature.com/articles/d41586-023-00073-6&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Nature&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Past vote data outperformed the polls. How did it go so wrong?</title>
      <link>https://solmessing.netlify.app/post/what-the-polls-got-wrong-in-2020/</link>
      <pubDate>Sun, 08 Nov 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/what-the-polls-got-wrong-in-2020/</guid>
      <description>&lt;p&gt;It’s becoming clear that the 2020 polls underestimated Trump’s support by anywhere from a 4-8 point margin depending on your accounting&amp;ndash;a significantly worse miss than in 2016, when &lt;a href=&#34;https://fivethirtyeight.com/features/the-polls-are-all-right/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;state polls were off but the national polls did relatively well&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In fact, this year we were better off using projections based on past vote history in each state to predict how things would go in battleground states, as I&amp;rsquo;ll show below.&lt;/p&gt;
&lt;p&gt;But I also want to start to ask questions about what happened this time around. The polling from 2018 looked encouraging, convincing many pollsters that the post-2016 reckoning had fixed many issues called out in the &lt;a href=&#34;https://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;2016 AAPOR report on election polling&lt;/a&gt;. After 2018, FiveThirtyEight wrote that the &lt;a href=&#34;https://fivethirtyeight.com/features/the-polls-are-all-right/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;Polls are Alright&amp;rdquo;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;But the second Miami-Dade reported results from the 2020 election, we knew something was probably wrong with the 2020 polls.&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Here&amp;#39;s another chart of polls vs. returns that splits the data by how much the polls underestimated Trump. One place where the polls most underestimated Trump is Wisconsin (off by -9 points). Note returns are not yet verified and states are still finalizing their counts. &lt;a href=&#34;https://t.co/iM8mjqoAuK&#34;&gt;pic.twitter.com/iM8mjqoAuK&lt;/a&gt;&lt;/p&gt;&amp;mdash; Stefan (@stefanjwojcik) &lt;a href=&#34;https://twitter.com/stefanjwojcik/status/1325786708022079488?ref_src=twsrc%5Etfw&#34;&gt;November 9, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;As Stefan notes (we worked together at Pew Research Center&amp;rsquo;s Data Labs), the error seems slightly lower in key battleground states, though the polls missed big in WI, perhaps in part due to its horrifically bad voter file data.&lt;/p&gt;
&lt;p&gt;Unlike 2016, both state and national polls appeared to underestimate Trump&amp;rsquo;s support, as this early (Nov 7) analysis from &lt;a href=&#34;https://twitter.com/thomasjwood&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tom Wood&lt;/a&gt; shows:&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Current as to the afternoon on the 7th, and with Senate results too. &lt;a href=&#34;https://t.co/EGZGarRPNj&#34;&gt;pic.twitter.com/EGZGarRPNj&lt;/a&gt;&lt;/p&gt;&amp;mdash; Tom Wood (@thomasjwood) &lt;a href=&#34;https://twitter.com/thomasjwood/status/1325199348553162752?ref_src=twsrc%5Etfw&#34;&gt;November 7, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;!-- [![normal](/img/TWpollingerror.jpeg)](https://twitter.com/thomasjwood/status/1325199348553162752) --&gt;
&lt;h2 id=&#34;polling-versus-past-votes&#34;&gt;Polling versus past votes&lt;/h2&gt;
&lt;p&gt;Perhaps what surprised me the most about polling this time around was when I went to evaluate some election projections I put together in April that we used internally at Acronym to help evaluate where we might want to spend. I pulled in the &lt;a href=&#34;https://www.nytimes.com/live/2020/presidential-polls-trump-biden&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NYTimes polling averages&lt;/a&gt; and compared them with the latest state-level presidential results from the AP. I then did the same for the April projections. Turns out the projections were significantly more accurate than the polling averages:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/PollingVSPastVoteProj.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;We used these projections, and other extant data (including the fact that there are two Senate races in play), when making what turned out to be a very lucky decision to start spending money in Georgia. We were one of the biggest and earliest spenders in that race.&lt;/p&gt;
&lt;p&gt;What are these projections? I simply took the last two state-level Presidential and U.S. House election totals, estimated each state&amp;rsquo;s &amp;ldquo;trajectory,&amp;rdquo; and added that to each state&amp;rsquo;s Democratic margin from the previous cycle.&lt;/p&gt;
&lt;p&gt;(Note that I also weighted 60-40 toward the Presidential results, and slightly regularized both the latest margin and the trajectory toward zero.)&lt;/p&gt;
&lt;p&gt;Informing this approach is work from &lt;a href=&#34;https://catalist.us/yair-ghitza-phd/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Yair Ghitza&lt;/a&gt; describing what went wrong in 2016, which suggested polarization and other state-level trends would continue, in addition to national trends or &amp;ldquo;uniform swing.&amp;rdquo; &lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;This paper from &lt;a href=&#34;https://twitter.com/SimonJackman?ref_src=twsrc%5Etfw&#34;&gt;@SimonJackman&lt;/a&gt;  also deserves a big hat tip &lt;a href=&#34;https://t.co/CTTpYDPwl2&#34;&gt;https://t.co/CTTpYDPwl2&lt;/a&gt;&lt;/p&gt;&amp;mdash; Sol Messing (@SolomonMg) &lt;a href=&#34;https://twitter.com/SolomonMg/status/1325564912798752773?ref_src=twsrc%5Etfw&#34;&gt;November 8, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;I should note that this may only have worked because of something peculiar about this election cycle&amp;ndash;I haven&amp;rsquo;t gone an back-tested this approach or anything like that.&lt;/p&gt;
&lt;p&gt;Seems I was not the only one who noticed this kind of pattern:&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;A similar observation from &lt;a href=&#34;https://twitter.com/gelliottmorris?ref_src=twsrc%5Etfw&#34;&gt;@gelliottmorris&lt;/a&gt; &lt;a href=&#34;https://t.co/XSUAhGBZfb&#34;&gt;https://t.co/XSUAhGBZfb&lt;/a&gt;&lt;/p&gt;&amp;mdash; Sol Messing (@SolomonMg) &lt;a href=&#34;https://twitter.com/SolomonMg/status/1325522770890027008?ref_src=twsrc%5Etfw&#34;&gt;November 8, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;h2 id=&#34;what-went-wrong-the-usual-suspects&#34;&gt;What went wrong: The Usual Suspects&lt;/h2&gt;
&lt;p&gt;Humble-brag aside, it&amp;rsquo;s worth asking what might have gone wrong with polling in 2020?&lt;/p&gt;
&lt;p&gt;The &lt;a href=&#34;https://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;2016 AAPOR report on election polling&lt;/a&gt; provides some guidance for how we might start to examine issues with the 2020 polls.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Undecided voters&lt;/strong&gt;: Undecideds broke toward Trump late in the election in 2016&amp;ndash;polls found as many as 13 percent of voters were &lt;a href=&#34;https://fivethirtyeight.com/features/the-invisible-undecided-voter/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;undecided on election day or planned to vote for a third party&lt;/a&gt;. According to Poynter, there were &lt;a href=&#34;https://www.poynter.org/fact-checking/2020/2020-is-not-like-2016-heres-whats-different/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;half as many of these voters in 2020&lt;/a&gt;, so this is unlikely to be as big a factor as in 2016.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Low education non-response &amp;amp; adjustment&lt;/strong&gt;: In 2016, individuals lower levels of education were much less likely to answer polls but still voted, and broke for Trump. The national polls adjusted for this but state level polls did not, which is partially why forecasting models that rely on state-level polls missed so hard.&lt;/p&gt;
&lt;p&gt;While many state-level pollsters did this in 2020, Pew Research Center still &lt;a href=&#34;https://www.pewresearch.org/methods/2020/08/18/a-resource-for-state-preelection-polling/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;found problems with state level polling this time around&lt;/a&gt;, for example failing to adjust for race and education simultaneously&amp;ndash;non-college whites are far more likely to support Trump than non-college non-whites.&lt;/p&gt;
&lt;p&gt;What&amp;rsquo;s more, pollsters adjusted only for college/non-college, which may not have been enough. They might need to use more fine grained adjustment&amp;ndash;accounting for whether respondents have a high school degree and a college degree.  Also error/missing data when people complete education in a survey means trouble if you want to fully fix the issue.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Volunteerism &amp;amp; civic engagement&lt;/strong&gt;: Even if you adjust for low levels of non-response among individuals with lower education, pollsters still may have problems reaching &lt;a href=&#34;https://www.pewresearch.org/fact-tank/2015/07/21/the-challenges-of-polling-when-fewer-people-are-available-to-be-polled/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;low civic engagement voters, a bias that seems to persist even after modeling/weighting adjustments&lt;/a&gt;. In the past this hasn&amp;rsquo;t mattered as much, but these folks may be showing up to the polls for Trump.&lt;/p&gt;
&lt;h2 id=&#34;other-potential-factors&#34;&gt;Other Potential Factors&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Likely voter models&lt;/strong&gt;: This is difficult to fully unpack since each polling house does this slightly differently and not all publish their methods—some ask a battery of voter questions, some use models, some recruit off the voter file. But there’s only a weak relationship between who votes and who scores high on the likely voter battery. To make matters worse, 2020 was a very high-turnout election, which could have introduced even more instability into likely voter models.&lt;/p&gt;
&lt;p&gt;Another important point from Peter Suzman is that likely voter screens could have inflated estimates of Dem turnout if they asked if respondents had already voted&amp;mdash;it was Democrats who voted early.&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Easily. An &amp;quot;unlikely&amp;quot; voter (by some screen) on the Dem side that has already voted gets counted; his/her exact counterpart on the GOP side that votes on election day doesn&amp;#39;t. &lt;br&gt;I commented before the election on the possibility that polls would skew Dem because of this.&lt;/p&gt;&amp;mdash; Peter Suzman (@Biomaven) &lt;a href=&#34;https://twitter.com/Biomaven/status/1325545770230161408?ref_src=twsrc%5Etfw&#34;&gt;November 8, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;However, that would only explain error in likely voter models, not polling based on registered voters, which also seemed to miss big this cycle, as I pointed out:&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;Still interesting but an issue w this explanation is that while it could interact with likely voter models, it’s unlikely to explain the error in registered voter samples which rely on the voter file over likely voter question batteries. (HT an old friend) &lt;a href=&#34;https://t.co/F2xciDV7vr&#34;&gt;https://t.co/F2xciDV7vr&lt;/a&gt;&lt;/p&gt;&amp;mdash; Sol Messing (@SolomonMg) &lt;a href=&#34;https://twitter.com/SolomonMg/status/1325605403636486146?ref_src=twsrc%5Etfw&#34;&gt;November 9, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;&lt;strong&gt;COVID-19&lt;/strong&gt;: I wrote about &lt;a href=&#34;https://solomonmg.github.io/post/trumps-chances-are-better-than-they-look/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this back in June&lt;/a&gt;. It&amp;rsquo;s possible that COVID-19 made lines long and kept people home in urban areas and non-white communities. Yes we had record turnout but all it takes is a few percent of people who encounter a bit of voting friction, who fail to register in person, don’t get in person canvassing/gotv contact, don’t vote by mail early, and/or don’t vote in vote in person.&lt;/p&gt;
&lt;p&gt;At the same time, &lt;a href=&#34;https://www.nytimes.com/2020/11/10/upshot/polls-what-went-wrong.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;David Shor points out&lt;/a&gt; in a piece by Nate Cohn at the New York Times, that &amp;ldquo;&amp;hellip;after lockdown, Democrats just started taking surveys, because they were locked at home and didn’t have anything else to do.&amp;rdquo;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/DavidShorOLNCNYT.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Without Dems doing the usual in-person registration drives, organizing, canvassing, etc. plus long lines in the hardest hit areas, and with Democrats taking surveys at unusually high rates, we might expect to see Trump overperform in areas hit hardest by COVID-19.&lt;/p&gt;
&lt;p&gt;And indeed the data show just that. &lt;a href=&#34;https://www.npr.org/sections/health-shots/2020/11/06/930897912/many-places-hard-hit-by-covid-19-leaned-more-toward-trump-in-2020-than-2016&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NPR has a nice visualization of this&lt;/a&gt;:&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;One potential explanation for Trump&amp;#39;s higher performance in areas hit hard by COVID is that by staying home, Dems didn&amp;#39;t do the usual in-person registration drives, organizing, canvassing, etc. NPR has a nice visualization of this &lt;a href=&#34;https://t.co/a8DeO27giE&#34;&gt;https://t.co/a8DeO27giE&lt;/a&gt; &lt;a href=&#34;https://t.co/1Kc9qNE4jJ&#34;&gt;pic.twitter.com/1Kc9qNE4jJ&lt;/a&gt;&lt;/p&gt;&amp;mdash; Sol Messing (@SolomonMg) &lt;a href=&#34;https://twitter.com/SolomonMg/status/1326348370869415937?ref_src=twsrc%5Etfw&#34;&gt;November 11, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;Another possibility is that shutdowns, school closings, and job losses stoked anger &amp;amp; resentment in centrist &amp;amp; right-leaning voters. I remember watching a local FB group quickly organize around the issue of school-openings and eventually morph into a hub for protests.&lt;/p&gt;
&lt;p&gt;EDIT: I took a look at his performance by the urbanicity and racial makeup of those counties and here&amp;rsquo;s what I found:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/share_over_2016_pct_nonwhite_covid_up_weighted.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;

















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/share_over_2016_pct_urban_covid_up_weighted.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Trump outperforms 2016 in non-white counties, and UNDER-performs in mostly-white counties. Same for more urban counties. That&amp;rsquo;s consistent w/ covid hitting non-white counties much harder in terms of registration, long-lines, and lower VBM rates.&lt;/p&gt;
&lt;p&gt;That seems to stand in sharp contrast to speculation that Trump would be hit hardest in areas where people are most likely to know someone with COVID.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Shy Trump voters&lt;/strong&gt;: There&amp;rsquo;s a hypothesis out there that people are embarrassed to admit that they would vote for Trump. The evidence for this is limited&amp;ndash;Kyle Dropp and co at Morning Consult did some experimental work on this and found that people were slightly more likely &lt;a href=&#34;https://morningconsult.com/form/shy-trump-2020/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;in the 2016 primaries (but NOT the General and not in 2020)&lt;/a&gt; to say that they would vote for Trump when answering via online survey compared with speaking with a live pollster over the phone. But they&amp;rsquo;ve done many follow-on surveys since and the pattern doesn&amp;rsquo;t persist.&lt;/p&gt;
&lt;blockquote class=&#34;twitter-tweet&#34;&gt;&lt;p lang=&#34;en&#34; dir=&#34;ltr&#34;&gt;One reason &amp;quot;shy Trump voters&amp;quot; doesn&amp;#39;t make as much sense as you&amp;#39;d think as an explanation for polling errors this year is that Republican candidates for Congress generally outperformed their polls by more than Trump did.&lt;/p&gt;&amp;mdash; Nate Silver (@NateSilver538) &lt;a href=&#34;https://twitter.com/NateSilver538/status/1324948324718436352?ref_src=twsrc%5Etfw&#34;&gt;November 7, 2020&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async src=&#34;https://platform.twitter.com/widgets.js&#34; charset=&#34;utf-8&#34;&gt;&lt;/script&gt;


&lt;p&gt;I am skeptical that this could be as much of a factor as some on social media seem to be claiming, but it&amp;rsquo;s hard to get good data to answer this question and acknowledge that absence of evidence is not evidence of absence. A number of commentators have claimed that since the polls underestimated support for all Republicans, this is an unlikely explanation.&lt;/p&gt;
&lt;p&gt;That sounds pretty air-tight at first glance but it&amp;rsquo;s possible that some undecideds, perhaps embarrassed about having Trump as a figurehead of the Republican party, refused to say with certainty who they would actually vote for. Nevertheless, based on the pattern of results we&amp;rsquo;ve seen so far, this really can&amp;rsquo;t explain very much of the polling error this time around.&lt;/p&gt;
&lt;h2 id=&#34;the-role-of-election-forecasts&#34;&gt;The Role of Election Forecasts&lt;/h2&gt;
&lt;p&gt;If you&amp;rsquo;re a forecaster, it&amp;rsquo;s very easy to look at all the polling data and come away with overconfident estimates of a candidate’s support. Many forecasters in 2016 did just that, failing to account for the fact that error between states and pollsters were likely correlated, and producing estimates that put Clinton’s chances above 95%.&lt;/p&gt;
&lt;p&gt;The Huffington Post famously &lt;a href=&#34;https://www.huffpost.com/entry/nate-silver-election-forecast_n_581e1c33e4b0d9ce6fbc6f7f&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;roasted FiveThirtyEight&lt;/a&gt; for trying to adjust for this state-level polling error the day before the 2016 election.&lt;/p&gt;
&lt;p&gt;But even when forecasters get it right, forecasting can create firm expectations that one candidate will win, which in 2016 was complicated by destiny-narrative driven by media coverage of election forecasting.&lt;/p&gt;
&lt;p&gt;Sean Westwood, Yph Lelkes and I recently published a &lt;a href=&#34;https://solomonmg.github.io/pdf/aggregator.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;research paper&lt;/a&gt; in the Journal of Politics showing just how much additional confidence forecasts give us, and wrote about the implications for the 2020 election in a recent &lt;a href=&#34;https://www.usatoday.com/story/opinion/2020/10/01/election-forecasts-can-wrong-you-still-need-vote-column/5857993002/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;USA Today op ed&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;I believe it was the sharp violation of expectations that was so disappointing to Clinton supporters and so invigorating for the MAGA crowd—the Washington elite had underestimated &amp;ldquo;real Americans&amp;rdquo; yet again.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Trump&#39;s chances are better than they look</title>
      <link>https://solmessing.netlify.app/post/trumps-chances-are-better-than-they-look/</link>
      <pubDate>Sat, 20 Jun 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/trumps-chances-are-better-than-they-look/</guid>
      <description>&lt;p&gt;According to the latest polling research, Trump’s chances of hanging on to power beyond 2020 look pretty dismal. Nate Cohn published an impressive battleground &lt;a href=&#34;https://www.nytimes.com/2020/06/25/upshot/poll-2020-biden-battlegrounds.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;poll from New York Times/Sienna&lt;/a&gt; showing Biden ahead of Trump by at least six points in pivotal states. The Economist’s forecast, powered by Elliott Morris and Andrew Gelman, is suggesting Biden is likely to get 64% of electoral college votes, and that if the election were held 100 times Biden would &lt;a href=&#34;https://statmodeling.stat.columbia.edu/2020/06/12/election-2020-is-coming-our-poll-aggregation-model-with-elliott-morris-of-the-economist/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;win 90 times to Trump’s 10&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;At this point I would like to remind you of that feeling you felt on election night 2016. When a month earlier, &lt;a href=&#34;https://www.cnn.com/2016/10/23/politics/hillary-clinton-donald-trump-presidential-polls/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CNN&amp;rsquo;s &amp;lsquo;Poll of Polls&amp;rsquo; had Clinton up by 9 points&lt;/a&gt; and two prominent forecasters put Clinton’s chances at 99%. Remember that?&lt;/p&gt;
&lt;p&gt;I could probably stop there, but I’m not going to because although we&amp;rsquo;ve fixed some of the issues from 2016, we have COVID-19. And COVID will mess with our election in ways very likely to hurt Democrats, and I know of no pollster factoring this into their method or likely voter model.&lt;/p&gt;
&lt;p&gt;After 2016, Sean Westwood, Yphtach Lelkes and I began a multi-year research project (recently published in the &lt;a href=&#34;https://www.journals.uchicago.edu/doi/abs/10.1086/708682?mobileUi=0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Journal of Politics&lt;/a&gt;) and found that when you have high confidence that one candidate will win, &lt;a href=&#34;https://solomonmg.github.io/project/projecting_confidence/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;you’re less likely to vote&lt;/a&gt;. The fact that everyone thought Clinton would win in 2016 shaped Comey&amp;rsquo;s decision to release his infamous letter that &lt;a href=&#34;https://fivethirtyeight.com/features/the-comey-letter-probably-cost-clinton-the-election/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;some believe cost Clinton the election&lt;/a&gt;, changed the way campaigns operated, and likely &lt;a href=&#34;https://solomonmg.github.io/pdf/aggregator.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;lowered Democratic turnout&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In addition to showing this in an experiment, one pattern that clearly pops out in the data we analyzed (ANES timeseries) is that people who think the leading candidate will win by quite a bit report voting at about a 3% lower rate. That&amp;rsquo;s in line with other research showing that early exit polls indicating one candidate is likely to win &lt;a href=&#34;https://www.jstor.org/preview-page/10.2307/2748722?seq=1&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;decrease turnout&lt;/a&gt;, and are more likely to &lt;a href=&#34;https://repository.upenn.edu/cgi/viewcontent.cgi?referer=&amp;amp;httpsredir=1&amp;amp;article=1018&amp;amp;context=asc_papers&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;affect Democrats&lt;/a&gt;. Yet this is by no means an upper bound&amp;mdash;one study found more decisive exit polling &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S0014292115000483&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;depressed turnout by 11 points&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/closerace_vote_anes.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;While it&amp;rsquo;s if anything a noisy indicator of the influence Clinton&amp;rsquo;s ostensible lead may have had on Democrats compared with Republicans, the proportion of Democrats who thought Clinton would ‘win by quite a bit’ was much higher in 2016 than for Republicans, and much higher than it&amp;rsquo;d been in many years.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/anes_turnout_closerace_mc_tall.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;To be clear, I no longer occupy the role of dispassionate observer&amp;ndash;I’m actively working in politics at the moment.&lt;/p&gt;
&lt;p&gt;So while I like seeing Biden up, let me explain exactly why the margins we’re seeing could be a polling mirage.&lt;/p&gt;
&lt;h2 id=&#34;covid-19&#34;&gt;COVID-19&lt;/h2&gt;
&lt;p&gt;Are pollsters accounting for the likely decline in urban turnout due to COVID-19? Not if they are assuming typical levels of turnout across urban and rural areas.&lt;/p&gt;
&lt;p&gt;Make no mistake, COVID-19 is already affecting the political process&amp;mdash;look at voter registration. As many colleagues who regularly deal with registration data have warned me, the usual rush of new voter registrations, often from young voters, have &amp;ldquo;fallen off a cliff.&amp;rdquo; Registration numbers started stronger than ever as the new year began, but as &lt;a href=&#34;https://fivethirtyeight.com/features/voter-registrations-are-way-way-down-during-the-pandemic/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;538 notes, fell to unprecedented levels in March&lt;/a&gt; as pandemic social distancing measures took effect.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://fivethirtyeight.com/features/voter-registrations-are-way-way-down-during-the-pandemic/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/538-voter-registrations-are-way-way-down-during-the-pandemic.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;So it&amp;rsquo;s already hurting Democrats in terms of new registrations, but what might all this mean on election day? At first blush, it may be tempting to say to yourself, “COVID is affecting old people more than the young, and they break conservative so the left is probably fine,” before feeling slightly ashamed that you’re thinking about strategic considerations before the loss of life and sadness this statement implies.&lt;/p&gt;
&lt;p&gt;Think a little deeper and you’ll likely realize that so far COVID-19 has affected left-leaning people in left-leaning places&amp;mdash;&lt;a href=&#34;https://www.npr.org/2020/04/12/832455226/what-coronavirus-exposes-about-americas-political-divide&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;non-White voters in urban areas&lt;/a&gt; far more than their suburban/rural counterparts. Even the recent &lt;a href=&#34;https://www.theatlantic.com/politics/archive/2020/06/coronavirus-surge-sun-belt-could-doom-trump/613495/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;surge in cases in sunbelt states&lt;/a&gt; is hitting urban and non-White regions hardest.&lt;/p&gt;
&lt;p&gt;What’s more, conservatives seem to be far more likely to be willing risk going out and about than liberals. A Pew study shows &lt;a href=&#34;https://www.pewresearch.org/fact-tank/2020/05/07/americans-remain-concerned-that-states-will-lift-restrictions-too-quickly-but-partisan-differences-widen/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Republicans are far more likely&lt;/a&gt; to support lifting COVID restrictions quickly than Democrats.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://www.pewresearch.org/fact-tank/2020/05/07/americans-remain-concerned-that-states-will-lift-restrictions-too-quickly-but-partisan-differences-widen/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/covid-partisan-differences-widen-Pew.png&#34; alt=&#34;wide&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;With a deadly pandemic raging, will urban and non-urban voters go to the polls at the usual rates?&lt;/p&gt;
&lt;p&gt;Post-pandemic primary voting has meant a vast reduction in the number of polling places and a big increase in mail-in-ballots. We&amp;rsquo;re seeing this in post-pandemic primaries like this Tuesday&amp;rsquo;s in &lt;a href=&#34;https://www.nytimes.com/2020/06/23/us/politics/kentucky-new-york-election-recap.html?action=click&amp;amp;module=Top%20Stories&amp;amp;pgtype=Homepage&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Kentucky, New York, and Virginia&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In New York&amp;rsquo;s primary, there were reports of &lt;a href=&#34;https://www.thecity.nyc/2020/6/23/21300471/nyc-primary-missing-ballots-busted-machines-pandemic&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;missing mail in ballots&lt;/a&gt;.  Kentucky also saw reports of &lt;a href=&#34;https://www.motherjones.com/politics/2020/06/kentucky-slashes-polling-places-voting-rights-mcgrath-booker-lebron-james/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;long lines that disportionately hit Black neighborhoods&lt;/a&gt;, in a primary that will determine the Democrat who runs against Senate Majority Leader Mitch McConnell.&lt;/p&gt;
&lt;p&gt;What at first looks like maybe a silver lining is the surge in voting by mail-in ballot. And while Trump sees mail-in ballots as a &lt;a href=&#34;https://www.politico.com/news/2020/06/19/trump-interview-mail-voting-329307&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;threat to his re-election&lt;/a&gt;, the evidence is far from clear that widespread voting by mail would hurt his chances.&lt;/p&gt;
&lt;p&gt;On the contrary, Stanford’s Andy Hall estimates that universal vote by mail &lt;a href=&#34;https://www.pnas.org/content/117/25/14052&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;should have no impact on either party&amp;rsquo;s vote share&lt;/a&gt;. However, as they note, vote by mail may very well have a disparate impact on minority voters, and their estimates assume that every voter is mailed a ballot, rather than needing to opt-in to voting by mail.&lt;/p&gt;
&lt;p&gt;And just today, the Supreme Court &lt;a href=&#34;https://www.nytimes.com/2020/06/26/us/supreme-court-texas-vote-by-mail.html?action=click&amp;amp;module=Top%20Stories&amp;amp;pgtype=Homepage&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;denied an emergency request&lt;/a&gt; to allow all citizens in Texas to vote by mail. That’s not the last word, but conservatives are actively fighting measures like this one, which would have made it far easier to prepare to handle a deluge of mail-in ballots in the fall.&lt;/p&gt;
&lt;p&gt;Furthermore, we’re already seeing evidence in the primaries of &lt;a href=&#34;https://www.nytimes.com/2020/03/09/us/virus-election-voting.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;poll-workers failing to show up&lt;/a&gt;, lengthening the already long lines in urban areas that discourage voters.&lt;/p&gt;
&lt;p&gt;If &lt;a href=&#34;https://faculty.ucmerced.edu/thansford/Articles/The%20Republicans%20Should%20Pray%20for%20Rain%20-%20Weather,%20Turnour,%20and%20Voting%20in%20U.S.%20Presidential%20Elections.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;a little bit of rain can depress turnout in urban areas&lt;/a&gt;, fear of a deadly pandemic that spreads when you’re standing in line seems likely to as well.&lt;/p&gt;
&lt;p&gt;What’s more, it’s going to take longer to count mail in ballots, and there will almost certainly be confusion about results &lt;a href=&#34;https://www.poynter.org/fact-checking/2020/be-patient-on-election-night-2020-counting-the-returns-will-take-time/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;as Poynter recently noted&lt;/a&gt;. Based on the President’s rhetoric around voting by mail, there will almost certainly be legal disputes about the legitimacy of certain results if not the election writ large.&lt;/p&gt;
&lt;p&gt;Buckle up.&lt;/p&gt;
&lt;h2 id=&#34;things-change&#34;&gt;Things Change&lt;/h2&gt;
&lt;p&gt;Six months ago the big story was the prospect of war with Iran after Trump killed Sulamani. The political world is fundamentally different now and it&amp;rsquo;s more than possible that something important will happen between now and election day with political consequences.&lt;/p&gt;
&lt;p&gt;Does that matter? Andrew Gelman (yes, the same) and Gary King have a paper suggesting it doesn&amp;rsquo;t&amp;mdash;showing that we can &lt;a href=&#34;https://www.jstor.org/preview-page/10.2307/194212?seq=1&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;predict elections remarkably well despite how much polls fluctuate&lt;/a&gt;. &amp;ldquo;Thus, the general campaign for president seems irrelevant to the outcome &amp;hellip; despite all the media coverage of campaign strategy&amp;hellip; &lt;em&gt;except in very close elections&lt;/em&gt;.&amp;rdquo; And &lt;a href=&#34;https://pollyvote.com/en/components/models/retrospective/fundamentals-plus-models/time-for-change-model/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Alan Abramowitz&amp;rsquo;s forecasting model&lt;/a&gt; which is the kind of model they are referencing and which has done extremely well in the past, has &lt;a href=&#34;https://www.rasmussenreports.com/public_content/political_commentary/commentary_by_alan_i_abramowitz/assessing_trump_s_chances_forecasting_the_2020_presidential_election&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Trump&amp;rsquo;s chances in 2020 nearly even&lt;/a&gt; (though both the economy and Trump&amp;rsquo;s polling numbers have suffered since).&lt;/p&gt;
&lt;p&gt;So even if you&amp;rsquo;re one of those people who think that in general the ebb and flow of historical events largely does not impact U.S. elections, it may matter more in 2020 than in a typical year, and that&amp;rsquo;s before you even factor in a global pandemic that has upended life in America.&lt;/p&gt;
&lt;p&gt;OK, but how many people are really undecided about Trump? When asked who they’d vote for, 8 percent of people in Nate Cohn’s poll said something other than Biden or Trump.&lt;/p&gt;
&lt;p&gt;According to the American Association of Public Opinion Research 2016 post-mortem, &lt;a href=&#34;https://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;a tsunami of undecided voters went to Trump&lt;/a&gt;, which was a major reason we thought Clinton was going to win in 2016. One controversial possibility is that some of these undecided voters were actually &amp;ldquo;&lt;a href=&#34;https://morningconsult.com/2016/11/03/shy-trump-social-desirability-undercover-voter-study/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;shy Trump supporters&lt;/a&gt;,&amp;rdquo; which might explain the swing. Of course, another controversial possibility is that &lt;a href=&#34;https://fivethirtyeight.com/features/the-comey-letter-probably-cost-clinton-the-election/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;the Comey letter cost Clinton the election&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Regardless, Trump is more well-known now than in 2016 and there &lt;em&gt;are&lt;/em&gt; fewer undecideds this time around. But we have no clue how these folks will break in 2020, and in 2016 they broke for Trump.&lt;/p&gt;
&lt;h2 id=&#34;correcting-for-political-engagement&#34;&gt;Correcting for Political Engagement&lt;/h2&gt;
&lt;p&gt;That 8 percent undecided number above may very well be an underestimate. Arguably the biggest problem in survey research today is that you can&amp;rsquo;t fully adjust for the &lt;a href=&#34;https://www.pewresearch.org/fact-tank/2015/07/21/the-challenges-of-polling-when-fewer-people-are-available-to-be-polled/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;bias toward high political knowledge respondents&lt;/a&gt;. And low political knowledge voters are more likely than others to be undecided.&lt;/p&gt;
&lt;h2 id=&#34;education&#34;&gt;Education&lt;/h2&gt;
&lt;p&gt;Likewise, it&amp;rsquo;s difficult to survey Americans with low education. The &lt;strong&gt;vast majority&lt;/strong&gt; of polls fail to recruit a representative swath of these potential voters and cannot fully adjust away the bias.&lt;/p&gt;
&lt;p&gt;OK so what?&lt;/p&gt;
&lt;p&gt;No election has &lt;a href=&#34;https://www.pewresearch.org/fact-tank/2016/11/09/behind-trumps-victory-divisions-by-race-gender-education/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;split on education like 2016&lt;/a&gt; going back to the beginning of Pew Research Center&amp;rsquo;s data on this in 1980. Non-college Whites voted for Trump over their college educated counterparts by a &lt;strong&gt;35 point margin&lt;/strong&gt;.  And the best retrospective analyses show that his biggest gains have &lt;a href=&#34;https://williammarble.co/docs/vb.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;come from low-education White moderates in battleground states&lt;/a&gt; (and &lt;em&gt;not&lt;/em&gt; as many have presumed, from those with conservative views on race and immigration, across the educational spectrum).&lt;/p&gt;
&lt;p&gt;Many 2016 polls did not adjust their samples to account for education&amp;mdash;something that mattered far less in the past and something not easy to do correctly. They systematically underestimated Trump’s support in part because of this issue.&lt;/p&gt;
&lt;p&gt;Although the NYT/Sienna poll and now many others do target and weight by education to increase representativeness, the &lt;a href=&#34;https://int.nyt.com/data/documenttools/nyt-siena-poll-methodology-june-2020/f6f533b4d07f4cbe/full.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;methodology&lt;/a&gt; shows this poll (along with most) lump together everyone without a college degree. While the &lt;a href=&#34;https://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;AAPOR report concludes this may be ok&lt;/a&gt;, Trump&amp;rsquo;s support does appear to increase as education decreases, which means failing to disaggregate &amp;ldquo;no high school degree,&amp;rdquo; &amp;ldquo;high school degree,&amp;rdquo; and &amp;ldquo;some college&amp;rdquo; when adjusting for education may very well result in some bias in favor of Trump.&lt;/p&gt;
&lt;h2 id=&#34;higher-error-in-subnational-polls&#34;&gt;Higher Error in Subnational Polls&lt;/h2&gt;
&lt;p&gt;The NYT/Sienna poll is one of the best subnational polls out there, but the error in battleground polls like this is generally &lt;a href=&#34;https://yougov.co.uk/topics/politics/articles-reports/2016/11/11/first-thoughts-polling-problems-2016-us-elections&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;higher than national polls&lt;/a&gt;. It&amp;rsquo;s harder to reach the right mix of people in individual states in a short period of time which increases the error.  By error, I mean the actual error in predicting presidential vote share, not the reported &amp;ldquo;margin of error,&amp;rdquo; which is usually &lt;a href=&#34;https://www.nytimes.com/2016/10/06/upshot/when-you-hear-the-margin-of-error-is-plus-or-minus-3-percent-think-7-instead.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;around half the actual error&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The reported margin of error here is about 2%, so doubling that, 4%, plus the 8 percent who didn&amp;rsquo;t say Biden or Trump means there may be 12% wiggle room, possibly more.&lt;/p&gt;
&lt;h2 id=&#34;issues-with-the-voter-file&#34;&gt;Issues with the Voter File&lt;/h2&gt;
&lt;p&gt;The way pollsters recruit people for their survey has a huge impact on accuracy. If you don’t get data from the right mix of people you’re not going to get a good sense of which candidate is ahead, and you can only get so much juice out of adjusting your polls using approaches like weighting.&lt;/p&gt;
&lt;p&gt;Pollsters often use random-digit dialing to get a representative sample, but many of the best election surveys run today are now conducted by calling people from the voter file. The &lt;a href=&#34;https://www.nytimes.com/2018/09/06/upshot/live-poll-explainer.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Times used the voter file&lt;/a&gt; in part so they could poll congressional districts, which are drawn in such idiosyncratic shapes that they don’t line up with area codes nor almost any other data set with phone numbers.&lt;/p&gt;
&lt;p&gt;The Times uses the voter file to target specific subsets of the population that are hard to reach, such as low-education voters. Unfortunately, running a voter-file based poll may still not get enough low-education voters&amp;mdash;as &lt;a href=&#34;https://www.pewresearch.org/methods/2018/10/09/performance-of-the-samples/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Pew Research Center’s voter file study&lt;/a&gt; showed (it used the same voter file vendor as does the Times&amp;mdash;L2). So you have to rely on statistical adjustment, increasing error.&lt;/p&gt;
&lt;p&gt;What’s more, in that Pew study only &lt;a href=&#34;https://www.pewresearch.org/methods/2018/10/09/comparing-survey-sampling-strategies-random-digit-dial-vs-voter-files/#overview-of-study-methodology&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;62% of respondents who answered on a cell phone&lt;/a&gt; were the actual person on the voter file. And these quality issues vary a lot by state&amp;mdash;remember, the file is first gathered by the secretary of state and is subject to local laws and regulations. For example, Wisconsin’s voter file is notoriously bad.&lt;/p&gt;
&lt;p&gt;All this increases total survey error and the chance that systematic biases will creep in.&lt;/p&gt;
&lt;p&gt;While the polling does indeed suggest better news than if it showed Trump ahead, this is still very likely a highly competitive race.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Projecting Confidence</title>
      <link>https://solmessing.netlify.app/project/projecting_confidence/</link>
      <pubDate>Mon, 18 May 2020 22:48:29 -0500</pubDate>
      <guid>https://solmessing.netlify.app/project/projecting_confidence/</guid>
      <description>&lt;p&gt;Inspired by Donald Trump&amp;rsquo;s surprise victory over Hillary Clinton in the 2016 general election, &lt;a href=&#34;https://www.dartmouth.edu/~seanjwestwood/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Sean Westwood&lt;/a&gt;, &lt;a href=&#34;http://ylelkes.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Yphtach Lelkes&lt;/a&gt; and I set out to interrogate the question of whether elecion forecasts&amp;mdash;particularly probablistic forecasts&amp;mdash;might create a sense of inevitability, and ultimately lead people to stay home on election day.&lt;/p&gt;
&lt;p&gt;Clinton herself was quoted in &lt;a href=&#34;http://nymag.com/daily/intelligencer/2017/05/hillary-clinton-life-after-election.html?mid=nymag_press&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;New York Magazine&lt;/a&gt; after the election:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;I had people literally seeking absolution&amp;hellip; ‘&lt;em&gt;I’m so sorry I didn’t vote. I didn’t think you needed me.&lt;/em&gt;’ I don’t know how we’ll ever calculate how many people thought it was in the bag, because the percentages kept being thrown at people — ‘&lt;em&gt;Oh, she has an 88 percent chance to win!&lt;/em&gt;’&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;&lt;strong&gt;Is it plausible that forecasting could have affected the election?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;For this phenomena to affect an election, it must:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;be visible in the media so it reaches potential voters,&lt;/li&gt;
&lt;li&gt;depress turnout, and&lt;/li&gt;
&lt;li&gt;affect one side more than the other. In the case of 2016, that means affecting Clinton&amp;rsquo;s supporters (and/or Clinton campaigners) more than Trump&amp;rsquo;s.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;We found evidence for all of the above. First, witness the rise of forecasts since 2008, when FiveThirtyEight first came on the scene:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/forecast_google_news.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;What&amp;rsquo;s more, there is good evidence that one side will be more affected. Our research (see results below) suggests that &lt;em&gt;candidate who is ahead in the polls is more affected&lt;/em&gt; by probablistic forecasts. In 2016, that was Hillary.&lt;/p&gt;
&lt;p&gt;And irrespective of 2016, it&amp;rsquo;s outlets with a &lt;em&gt;left-leaning audience&lt;/em&gt; that publish and cover election forecasts. The websites that present their poll aggregation results in terms of probabilities have left-leaning (negative) social media audiences&amp;mdash;only realclearpolitics.com, which doesn’t emphasize win-probabilities, has a conservative audience:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/bma_science_alignment.png&#34; alt=&#34;half&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;These data come from the average self-reported ideology of people who share links to various sites hosting poll-aggregators on Facebook, data that come from &lt;a href=&#34;http://science.sciencemag.org/content/early/2015/05/06/science.aaa1160.full&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this paper&lt;/a&gt;’s &lt;a href=&#34;http://dx.doi.org/10.7910/DVN/LDJ7MS&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;replication materials&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;When you look at the balance of coverage of probabilistic forecasts on major television broadcasts, there is more coverage on MSNBC, which has a more liberal audience.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/msnbc_mentions.png&#34; alt=&#34;half&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;How much influence do forecasters really have?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;It&amp;rsquo;s increadibly difficult to tease out when one media outlet is influencing another. However, a freak event in 2018 allows us to get some traction on this question, and suggests that FiveThirtyEight&amp;rsquo;s 2018 coverage was highly influential.&lt;/p&gt;
&lt;p&gt;After FiveThirtyEight&amp;rsquo;s real-time forecast suddenely moved the the GOP&amp;rsquo;s odds of taking the House from single digits to about 60% at around 8:15PM, PredictIt&amp;rsquo;s odds on the GOP rose above 50-50, &amp;amp; &lt;em&gt;U.S. government bond yields rose 2-4 basis points.&lt;/em&gt;  FiveThirtyEight then altered it&amp;rsquo;s prediction system and the markets calmed down.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/538-markets.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;This spike seems to have occurred because a number of big, &lt;a href=&#34;https://fivethirtyeight.com/live-blog/2018-election-results-coverage/#3495&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Republican-dominated districts started reporting returns before those that went toward Democrats&lt;/a&gt; and because it was making inferences from partial vote counts:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/538realtimepolling.jpg&#34; alt=&#34;half&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;This was &lt;a href=&#34;https://ftalphaville.ft.com/2018/11/07/1541617447000/Debt-markets-let-us-know-what-they-think-about-Republicans-last-night/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;first reported by Colby Smith &amp;amp; Brian Greeley of FT.com&lt;/a&gt;. They report that because markets expected to see more inflation under a Republican House (high spending, low taxes) the U.S. Bond yield rose.&lt;/p&gt;
&lt;p&gt;Was this just a correlation? Possibly, but there was pretty much nothing else happening in the U.S., and it was like 1 am in Europe, as pointed out in the FT.com piece above.&lt;/p&gt;
&lt;p&gt;Josh Tucker suggested that &lt;a href=&#34;http://themonkeycage.org/2012/10/convergence-between-polls-and-prediction-markets-in-us-presidential-election/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;538 might be driving prediction markets&lt;/a&gt; back in 2012 in a Monkey Cage blogpost.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Our research on forecasting and perception&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://solmessing.netlify.app/pdf/aggregator.pdf&#34;&gt;Our research&lt;/a&gt; shows that probablistic election forecasts make a race look less competitive. Participants in a national probability survey-experiment were substantially more certain that one candidate would win a hypothetical race after seeing a probablistic forecast than after seeing the equivalent vote share estimate and margin of error. This is a big effect&amp;mdash;those are confidence intervals not standard errors, with p-values below $$10^{-11}$$.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/certaintyc.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why do people do this?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;More research is needed here but we do have some leads. First, small differences in the election metric most familiar to the public—vote share estimates—generally correspond to very large differences in the probability of a candidate’s chance of victory.&lt;/p&gt;
&lt;p&gt;Andy Gelman referenced this in passing in a &lt;a href=&#34;https://andrewgelman.com/2012/10/22/is-it-meaningful-to-talk-about-a-probability-of-65-7-that-obama-will-win-the-election/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;2012 blogpost&lt;/a&gt; questioning the decimal precision (0.1 percent) that 538 used to communicate its forecast on its website:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;That’s right: a change in 0.1 of win probability corresponds to a 0.004 percentage point share of the two-party vote. I can’t see that it can possibly make sense to imagine an election forecast with that level of precision&amp;hellip;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Second, people sometimes confuse probabilistic forecasts with vote share projections, and incorrectly conclude that a candidate is projected to say win 85% percent of the vote, rather than to having an 85% chance of winning the election. About 1 in 10 peope did this in our experiment.&lt;/p&gt;
&lt;p&gt;As &lt;a href=&#34;https://twitter.com/jbenton/status/1059898288139354112&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Joshua Benton pointed out in a tweet&lt;/a&gt;, TalkingPointsMemo.com &lt;a href=&#34;https://talkingpointsmemo.com/news/issa-calls-race-early&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;made this very mistake&lt;/a&gt;:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/TPMCorrection.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Finally, people tend to think in qualitative terms about the probability of events {%cite sunstein2002probability%}, {%cite keren1991calibration%}. An 85% likelihood that something will happen means it&amp;rsquo;s going to happen. These studies may help explain why after the 2016 election, so many criticized forecasters for “getting it wrong” (see &lt;a href=&#34;https://www.nytimes.com/2016/11/10/technology/the-data-said-clinton-would-win-why-you-shouldnt-have-believed-it.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this&lt;/a&gt; and &lt;a href=&#34;http://www.slate.com/articles/news_and_politics/politics/2016/01/nate_silver_said_donald_trump_had_no_shot_where_did_he_go_wrong.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What about voting?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Perhaps most critically, we show that probabilistic forecasts showing more of a blowout can lower voting. In Study 1, we find limited evidence of this based on self reports. In Study 2, we show that when participants are faced with incentives designed to simulate real world voting, they are less likely to vote when probabilistic forecasts show higher odds of one candidate winning. Yet they are not responsive to changes in vote share.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/FT_18.01.03_prob_vote.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Could this actually affect real world voting?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Consider 2016&amp;mdash;an unusually high number of Democrats thought the leading candidate would &lt;em&gt;win by quite a bit&lt;/em&gt;:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/anes_turnout_closerace_mc_tall.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;And people who say the leading candidate will &lt;em&gt;win by quite a bit&lt;/em&gt; in pre-election polling are about three percentage points less likely to say they voted after the election than people who say it’s a close race. That’s after controlling for election year, prior turnout, and party identification.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/closerace_vote_anes.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;The data here are from the &lt;a href=&#34;https://electionstudies.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;American National Election Study (ANES)&lt;/a&gt; and go back to 1952.&lt;/p&gt;
&lt;p&gt;Past social science research also provides evidence that the perception of a close race boosts turnout. Some of the best evidence comes from work that analyzes the effects of releasing exit polling results before voting ends, which clearly removes uncertainty. Work examining the effects of East Coast television networks’ “early calls” for one candidate or another on West Coast turnout generally find small but substantively meaningful effects, despite the fact that these calls occur &lt;a href=&#34;https://repository.upenn.edu/cgi/viewcontent.cgi?article=1018&amp;amp;context=asc_papers&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;late on election day&lt;/a&gt;, see also &lt;a href=&#34;https://academic.oup.com/poq/article-pdf/50/3/331/5135691/50-3-331.pdf?casa_token=Ize8hznQUHAAAAAA:6tMTMfvGSN4tFXyyXbwkew4E47cLlCG8FegNu4ulkzqHE3hJZMzfurBb-Y1GWQcvLbZTYUysOMebxg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this&lt;/a&gt;. Similar work exploiting voting reform as a natural experiment &lt;a href=&#34;https://eprints.qut.edu.au/83681/1/1-s2.0-S0014292115000483-main.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;shows a full 11 percentage point decrease&lt;/a&gt; in turnout in the French overseas territories that voted after exit polls were released. These designs are not confounded with the tendency for campaigns to invest more in campaigns in competitive races.&lt;/p&gt;
&lt;p&gt;Researchers consistently find robust correlations between tighter elections and higher turnout &lt;a href=&#34;https://www.sciencedirect.com/science/article/abs/pii/S0261379405000910&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;see this&lt;/a&gt;; and &lt;a href=&#34;https://biopen.bi.no/bi-xmlui/bitstream/handle/11250/2389104/Geys_ES%202016.pdf?sequence=5&amp;amp;isAllowed=y&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this&lt;/a&gt; for reviews]. Furthermore, there is evidence from statistical models that &lt;a href=&#34;https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1052&amp;amp;context=poliscifacpub&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;prior election returns also explain turnout&lt;/a&gt; above and beyond campaign spending, particularly when good polling data is unavailable.&lt;/p&gt;
&lt;p&gt;Field experiments provide additional evidence that perceptions of higher electoral competition increases turnout. This work finds substantive effects on turnout when polling results showing a closer race are delivered &lt;a href=&#34;%28https://huber.research.yale.edu/materials/67_paper.pdf%29&#34;&gt;via telephone&lt;/a&gt; [among those who were reached] but null results when relying on &lt;a href=&#34;https://www.nber.org/papers/w23071&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;postcards&lt;/a&gt; to deliver closeness messages. Finally, one study conducted in the weeks leading up to the 2012 presidential election found higher rates of self-reported, post-election turnout when delivering ostensible polling results showing Obama neck-and-neck with Romney &lt;a href=&#34;http://www.aapor.org/AAPOR_Main/media/AnnualMeetingProceedings/2013/Session_C-5-1-Vannette.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;which was not consistent with the extant polling data showing a comfortable Obama lead&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Could this affect politicians as well?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Candidates’ perceptions of the closeness of an election can affect campaigning and representation {%cite enos2015campaign%}, {%cite Mutz:1997wy%}.&lt;/p&gt;
&lt;p&gt;These perceptions can also shape policy decisions—-for example, prior to the 2016 election, the Obama administration’s confidence in a Clinton victory was reportedly a factor in the muted response to &lt;a href=&#34;https://www.washingtonpost.com/graphics/2017/world/national-security/obama-putin-election-hacking/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Russian intervention in the election&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;And former FBI Director James Comey, because of his confidence in a Clinton victory, said he felt that it was his duty to write a letter to Congress on October 28 saying he was reopening the investigation into her emails. Comey explained his actions based on his certain belief in a Clinton win: &amp;lsquo;&amp;rsquo;[S]he&amp;rsquo;s gonna be elected president, and if I hide this from the American people, she&amp;rsquo;ll be illegitimate the moment she&amp;rsquo;s elected, the moment this comes out&amp;rsquo;&amp;rsquo; {%cite keneally_2018%}. Nate Silver at one point said &amp;lsquo;&amp;rsquo;&lt;a href=&#34;https://fivethirtyeight.com/features/the-comey-letter-probably-cost-clinton-the-election/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;the Comey letter probably cost Clinton the Election&lt;/a&gt;.&amp;rsquo;&amp;rsquo;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/ComeyABCCNNresize.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Media coverage&lt;/strong&gt;
&lt;a href=&#34;https://www.washingtonpost.com/news/politics/wp/2018/02/06/clintons-achilles-heel-in-2016-may-have-been-overconfidence/?utm_term=.619133ce9312&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Washington Post&lt;/a&gt;, &lt;a href=&#34;https://fivethirtyeight.com/features/politics-podcast-whats-so-wrong-with-nancy-pelosi/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FiveThirthyEight’s Politics Podcast&lt;/a&gt;, &lt;a href=&#34;http://nymag.com/intelligencer/2018/02/americans-dont-understand-election-probabilities.html?gtm=bottom&amp;amp;gtm=bottom&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;New York Magazine&lt;/a&gt;, &lt;a href=&#34;https://politicalwire.com/2018/02/06/election-forecasts-lower-voter-turnout/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Political Wire&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Facebook Condor URLs Data Release</title>
      <link>https://solmessing.netlify.app/project/condor_data_release/</link>
      <pubDate>Mon, 18 May 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/project/condor_data_release/</guid>
      <description>&lt;p&gt;On January 17, 2020 my team at Facebook launched one of the largest social science data sets ever constructed. It’s meant to facilitate research on misinformation from across the web, shared and spread on Facebook.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://solmessing.netlify.app/pdf/Facebook_DP_URLs_Dataset.pdf&#34;&gt;Full details on the release here&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;We also released the &lt;a href=&#34;https://github.com/facebookresearch/URL-Sanitization&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;URL santization framework&lt;/a&gt;, which I implemented (and which my SWE colleagues refactored).&lt;/p&gt;
&lt;p&gt;What makes this data release unprecedented is that it contains &lt;em&gt;exposure data&lt;/em&gt; describing external links that billions of users saw and read while using the site.&lt;/p&gt;
&lt;p&gt;The data set goes beyond URL-level data, breaking down exposure and interactions by month, country, age, gender, and in the U.S., political page affinity (see Barbera et al 2015).&lt;/p&gt;
&lt;p&gt;The data contain two tables: (1) a “URL attributes” table describing the 38 million URLs in the data set, including how many times users tagged those posts as containing misinformation, harassment, etc. and (2) a “breakdown” table, which aggregates counts of actions taken on urls, broken out by user demographics and URL attributes.&lt;/p&gt;
&lt;p&gt;The &lt;a href=&#34;https://solmessing.netlify.app/pdf/Facebook_DP_URLs_Dataset.pdf&#34;&gt;technical documentation&lt;/a&gt; reflects more work than most papers I&amp;rsquo;ve written: . This list of authors reflects the scale of this massive team effort, and that&amp;rsquo;s before you include increadibly helpful advice we got from a number of computer scientists in the academy listed in the acknowledgements.&lt;/p&gt;
&lt;p&gt;Perhaps most importantly, this release provides guarantees about anonymity in an incredibly rigorous way&amp;ndash;action-level differential privacy, while preserving more underlying signal in the data.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Impression of Influence</title>
      <link>https://solmessing.netlify.app/project/impression_of_influence/</link>
      <pubDate>Sun, 17 May 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/project/impression_of_influence/</guid>
      <description>&lt;p&gt;&lt;a href=&#34;https://solmessing.netlify.app/pdf/GrimmerWestwoodMessingBook.pdf&#34;&gt;&lt;strong&gt;The Impression of Influence: Legislator Communication, Representation, and Democratic Accountability&lt;/strong&gt;&lt;/a&gt;
&lt;strong&gt;Princeton University Press, 2015&lt;/strong&gt;. With Justin Grimmer and Sean
Westwood&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Media: &lt;a href=&#34;http://www.mischiefsoffaction.com/2015/01/its-frequency-not-size-compromise.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mischiefs of Faction&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Why Election Forecasting Matters</title>
      <link>https://solmessing.netlify.app/post/response-to-fivethirtyeights-podcast-about-our-paper-projecting-confidence/</link>
      <pubDate>Sun, 26 Apr 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/response-to-fivethirtyeights-podcast-about-our-paper-projecting-confidence/</guid>
      <description>&lt;p&gt;Do you remember the night of Nov 8, 2016? I was glued to election coverage and obsessively checking probabilistic forecasts, wondering whether Clinton might do &lt;em&gt;so well&lt;/em&gt; that she’d win in places like my home state of Arizona. Although FiveThirtyEight had Clinton&amp;rsquo;s chances at beating Trump at around 70%, most other forecasters had her at around 90%.&lt;/p&gt;
&lt;p&gt;When she lost, &lt;a href=&#34;https://www.nytimes.com/2016/11/10/us/politics/donald-trump-election-reaction.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;many on both sides of the aisle&lt;/a&gt; were shocked. My co-authors and I wondered if America&amp;rsquo;s seeming confidence in a Clinton victory wasn’t driven in part by increasing coverage of probabilistic forecasts. And, if a Clinton victory looked inevitable, what did that do to turnout?&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/forecast_google_news.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;We weren&amp;rsquo;t alone. Clinton herself was quoted in &lt;a href=&#34;http://nymag.com/daily/intelligencer/2017/05/hillary-clinton-life-after-election.html?mid=nymag_press&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;New York Magazine&lt;/a&gt; after the election:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;I had people literally seeking absolution&amp;hellip; ‘&lt;em&gt;I’m so sorry I didn’t vote. I didn’t think you needed me.&lt;/em&gt;’ I don’t know how we’ll ever calculate how many people thought it was in the bag, because the percentages kept being thrown at people — ‘&lt;em&gt;Oh, she has an 88 percent chance to win!&lt;/em&gt;’&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Enter our recent &lt;a href=&#34;http://www.pewresearch.org/fact-tank/2018/02/06/use-of-election-forecasts-in-campaign-coverage-can-confuse-voters-and-may-lower-turnout/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;blog post&lt;/a&gt; and &lt;a href=&#34;https://papers.ssrn.com/abstract=3117054&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paper released on SSRN&lt;/a&gt;, “Projecting confidence: How the probabilistic horse race confuses and de-mobilizes the public,” by Sean Westwood, Solomon Messing, and Yphtach Lelkes. While our work cannot definitively say whether probabilistic forecasts played a decisive role in the 2016 election, it does indeed show that compared to more conventional vote share projections, probabilistic forecasts can confuse people, can give people more confidence that the candidate depicted as being ahead will win, may decrease turnout, and that liberals in the U.S. are more likely to encounter them. We appreciate the media attention to this work, including coverage by the &lt;a href=&#34;https://www.washingtonpost.com/news/politics/wp/2018/02/06/clintons-achilles-heel-in-2016-may-have-been-overconfidence/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Washington Post&lt;/a&gt;, &lt;a href=&#34;http://nymag.com/daily/intelligencer/2018/02/americans-dont-understand-election-probabilities.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;New York Magazine&lt;/a&gt;, and the &lt;a href=&#34;https://politicalwire.com/2018/02/06/election-forecasts-lower-voter-turnout/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Political Wire&lt;/a&gt;. What&amp;rsquo;s more, FiveThirtyEight devoted much of their Feb. 12 &lt;a href=&#34;https://fivethirtyeight.com/features/politics-podcast-whats-so-wrong-with-nancy-pelosi/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Politics Podcast&lt;/a&gt; to a spirited, and at points critical discussion of our work. We are open to criticism and will respond to some of the questions raised in this post. Below, we’ll show that the evidence in our study and in other research is &lt;em&gt;not&lt;/em&gt; inconsistent with our headline, as the hosts suggest—we&amp;rsquo;ll detail the evidence that probabilistic forecasts &lt;em&gt;confuse&lt;/em&gt; people, irrespective of their technical accuracy. We’ll also discuss where we agree with the podcast hosts. Furthermore, we&amp;rsquo;ll discuss a few topics which, judging from the hosts discussion, may not have come through clearly enough in our paper. We&amp;rsquo;ll reiterate what this work contributes to social science—how the paper adds to our understanding of how people think about probabilistic forecasts and how they may decrease voting, particularly for the leading candidate&amp;rsquo;s supporters and among liberals in the U.S. We’ll then walk readers through the way we mapped vote share projections to probabilities in the study. Finally we’ll discuss why this work matters, and conclude by pointing out future research we’d like to see in this area.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What’s new here?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The research contains a number findings that are new to social science:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Presenting forecasted win-probabilities gives potential voters the impression that one candidate will win more decisively, compared with vote share projections (Study 1).&lt;/li&gt;
&lt;li&gt;Higher win probabilities, but &lt;em&gt;not&lt;/em&gt; vote share estimates, &lt;em&gt;decrease voting&lt;/em&gt; in the face of the trade-offs embedded in our election simulation (Study 2). This helps confirm the findings in Study 1 and adds to the evidence from past research that people vote at lower rates when they perceive an election to be uncompetitive.&lt;/li&gt;
&lt;li&gt;In 2016, probabilistic forecasts were covered more extensively than in the past and tended to be covered by outlets with more liberal audiences.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Where we agree&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;If what you care about is conveying an accurate sense of whether one candidate will win, probabilistic forecasts do this slightly better than vote share. And, they seem to give people an edge on accuracy when interpreting the vote share if your candidate is behind. Of course, people can be confused and still end up being accurate, as we&amp;rsquo;ll discuss below.&lt;/p&gt;
&lt;p&gt;We also agree that people often do not accurately judge the likelihood of victory after seeing a vote share projection. That makes sense because, as the study shows, people appear to largely ignore the margin of error, which they’d need to map between vote share estimates and win probabilities.&lt;/p&gt;
&lt;p&gt;We also agree that a lot of past work shows that people stay home when they think an election isn’t close. What we’re adding to that body of work is evidence that compared with vote share projections, probabilistic forecasts give people the impression that one candidate will win more decisively, and may thus &lt;em&gt;more powerfully&lt;/em&gt; affect turnout.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Does the evidence in our study contradict our headline?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Our headline isn’t about accuracy, it’s about &lt;em&gt;confusion&lt;/em&gt;. And the evidence from this research and past work taken as a whole suggests that probabilistic forecasts confuse people — something that came up at the end of segment — even if the result sometimes is technically higher accuracy.&lt;/p&gt;
&lt;p&gt;1. People in the study who saw only probabilistic forecasts were more likely to confuse probability and vote share. After seeing probabilistic forecasts, 8.6% of respondents mixed up vote share and probability, while only 0.6% of respondents did so after seeing vote share projections. We’re defining “mixed-up” as reporting the win-probability we provided as the vote share and vice-versa.&lt;/p&gt;
&lt;p&gt;2. Figure 2B (Study 1) shows that people get their candidate’s likelihood of winning very wrong, even when we explicitly told them the probability a candidate will win. It’s true that they got slightly closer with a probability forecast, but they are still far off.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/likelihood_loess.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Why might this be? A lot of past research and evidence suggests that people have trouble understanding probabilities, as noted at the end of the podcast. People have a tendency to think about &lt;a href=&#34;https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=1384&amp;amp;context=law_and_economics&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;probabilities in subjective terms&lt;/a&gt;, so they have trouble understanding &lt;a href=&#34;https://www.ncbi.nlm.nih.gov/pubmed/26161749&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;medical risks&lt;/a&gt; and even &lt;a href=&#34;http://pubman.mpdl.mpg.de/pubman/item/escidoc:2101059/component/escidoc:2101058/GG_30_Chance_2005.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;weather forecasts&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Nate Silver has himself &lt;a href=&#34;https://fivethirtyeight.com/features/the-media-has-a-probability-problem/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;made the argument&lt;/a&gt; that &lt;a href=&#34;https://www.nytimes.com/2016/11/10/technology/the-data-said-clinton-would-win-why-you-shouldnt-have-believed-it.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;the backlash&lt;/a&gt; we saw to data and analytics in the wake of the 2016 election is due in part to the media misunderstanding probabilistic forecasts.&lt;/p&gt;
&lt;p&gt;As the podcast hosts pointed out, people &lt;em&gt;underestimated&lt;/em&gt; the true likelihood of winning after seeing both probabilistic forecasts and vote share projections. It’s possible that people are skeptical of any probabilistic forecast in light of the 2016 election. It’s possible they interpreted the likelihood not as hard-nosed odds, but in rather subjective terms — &lt;em&gt;what might happen&lt;/em&gt;, &lt;a href=&#34;https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=1384&amp;amp;context=law_and_economics&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;consistent with past research&lt;/a&gt;. Regardless, they do not appear to reason about probability in a way that is consistent with how election forecasters define the probability of winning.&lt;/p&gt;
&lt;p&gt;3. Looking at how people reason about vote share — the way people have traditionally encountered polling data — it’s clear from our results that when a person’s candidate is ahead and they see a probabilistic forecast, they rather dramatically overestimate the vote share. On the other hand, when they are behind, they get closer to the right answer.&lt;/p&gt;
&lt;p&gt;But we know from past research that people have a “&lt;a href=&#34;http://journals.sagepub.com/doi/abs/10.1177/0956797609356421&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;wishful&lt;/a&gt; &lt;a href=&#34;http://www.tandfonline.com/doi/abs/10.1207/s15324834basp1304_6&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;thinking&lt;/a&gt;” &lt;a href=&#34;https://academic.oup.com/ijpor/article-abstract/9/2/105/713900&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;bias&lt;/a&gt;, meaning they say their candidate is doing better than polling data suggests. That’s why there’s a positive bias when people are evaluating how their candidate will do, according to Figure 2A (Study 1).&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/votea.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;The pattern in the data suggest that people are more accurate after seeing a probabilistic forecast for a losing candidate because of this effect, and &lt;em&gt;not&lt;/em&gt; necessarily because they better understand that candidate’s actual chances of victory.&lt;/p&gt;
&lt;p&gt;4. Perhaps even more importantly, &lt;em&gt;none&lt;/em&gt; of the results here changed when we excluded the margin of error from the projections we presented to people. That suggests that the public may not understand error in the same way that statisticians do, and therefore may not be well-equipped to understand what goes into changes in probabilistic forecast numbers. And of course, very small changes in vote share projection numbers and estimates of error correspond to much larger swings in probabilistic forecasts.&lt;/p&gt;
&lt;p&gt;5. Finally, as we point out in the paper, if probabilistic forecasters do not account for total error, they can really overestimate a candidate’s probability of winning. Of course, that’s because an estimate of the probability of victory bakes in estimates of error, which recent work has found is &lt;a href=&#34;http://www.stat.columbia.edu/~gelman/research/unpublished/polling-errors.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;often about twice as large as the estimates of sampling error&lt;/a&gt; provided in many polls.&lt;/p&gt;
&lt;p&gt;As &lt;a href=&#34;https://fivethirtyeight.com/features/the-polls-were-skewed-toward-democrats/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Nate Silver has alluded to&lt;/a&gt;, if the forecaster does not account unobserved error, including error that may be correlated across surveys — he/she will artificially inflate the estimated probability of victory or defeat. Of course, FiveThirtyEight &lt;em&gt;does&lt;/em&gt; attempt to account for this error, and released far more conservative forecasts than others in this space in 2016.&lt;/p&gt;
&lt;p&gt;Speaking in part to this issue, &lt;a href=&#34;http://www.stat.columbia.edu/~gelman/research/published/what_learned_in_2016_5.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Andrew Gelman and Julia Azari&lt;/a&gt; recently concluded that “polling uncertainty could best be expressed not by speculative win probabilities but rather by using the traditional estimate and margin of error.” They seemed to be speaking about other forecasters, and did not directly reference FiveThirtyEight.&lt;/p&gt;
&lt;p&gt;At the end of the day, it’s easy to see that a vote share projection of 55% means that “55% of the votes will go to Candidate A, according to our polling data and assumptions.” However, it’s less clear that an 87% win probability means that “if the election were held 1000 times, Candidate A would win 870 times, and lose 130 times, based on our polling data and assumptions.”&lt;/p&gt;
&lt;p&gt;And most critically, we show that probabilistic forecasts showing more of a blowout could potentially lower voting. In Study 1, we provide limited evidence of this based on self reports. In Study 2, we show that when participants are faced with incentives designed to simulate real world voting, they are less likely to vote when probabilistic forecasts show higher odds of one candidate winning. Yet they are not responsive to changes in vote share.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/FT_18.01.03_prob_vote.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What’s with our mapping between vote share and probability?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The podcast questions how a 55% vote share with a 2% margin of error is equivalent to an 87% win probability. This illustrates a common problem people have when trying to understand win probabilities — -it’s difficult to reason about the relationship between win-probabilities and vote share without actually running the numbers.&lt;/p&gt;
&lt;p&gt;You can express a projection as either (1) the average vote share (can be an electoral college vote share or the popular vote share)&lt;/p&gt;
&lt;p&gt;$$ \hat \mu_v = \frac{1}{N}\sum_{i}^{N}\bar x_i$$&lt;/p&gt;
&lt;p&gt;and margin of error&lt;/p&gt;
&lt;p&gt;$$\hat \mu_v \pm T^{0.975}_{df = N} \times \frac{ \hat \sigma_v}{\sqrt N}$$&lt;/p&gt;
&lt;p&gt;Here the average for each survey is $$\bar x_i$$, and there are $$N = 20$$ surveys.&lt;/p&gt;
&lt;p&gt;Or (2) the probability of winning — the probability that the vote share is greater than half, based on the observed vote share and standard error:&lt;/p&gt;
&lt;p&gt;$$1 - \Phi \left (\frac{.5 - \hat \mu_v}{\hat\sigma_v} \right )$$&lt;/p&gt;
&lt;p&gt;Going back to the example above, here’s the R code to generate those quantities:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;svy_mean&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.55&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;svy_SD&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.04483415&lt;/span&gt; &lt;span class=&#34;c1&#34;&gt;# see appendix &lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;N_svy&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;20&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;margin_of_error&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;qt&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.975&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;df&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;N_svy&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;svy_SD&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;sqrt&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N_svy&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;svy_mean&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;[1]&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.55&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;margin_of_error&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;[1]&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.02091224&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;prob_win&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;pnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;q&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.50&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mean&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;svy_mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;sd&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;svy_SD&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;prob_win&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;[1]&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.8676222&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;More details about this approach are in our appendix. This is similar to how the Princeton Election Consortium generated win probabilities in 2016.&lt;/p&gt;
&lt;p&gt;Of course, one can also use an approach based on simulation, as FiveThirtyEight does. In the case of the data we generated for our hypothetical election in Study 1, this approach is not necessary. However we recognize that in the case of real-world presidential elections, a simulation approach has clear advantages by virtue of allowing more flexible statistical assumptions and a better accounting of error.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why does this matter?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;To be clear, we are not analyzing real-world election returns. However, a lot of past &lt;a href=&#34;https://huber.research.yale.edu/materials/67_paper.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;research&lt;/a&gt; &lt;a href=&#34;http://www2.gsu.edu/~polsnn/priorbeliefs.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;shows&lt;/a&gt; that when people &lt;a href=&#34;https://www.jstor.org/stable/1953324?seq=1#page_scan_tab_contents&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;think&lt;/a&gt; an election is &lt;a href=&#34;https://repository.upenn.edu/cgi/viewcontent.cgi?referer=&amp;amp;httpsredir=1&amp;amp;article=1018&amp;amp;context=asc_papers&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;in the bag&lt;/a&gt;, they tend to &lt;a href=&#34;https://www.sciencedirect.com/science/article/pii/S0014292115000483&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;vote in real-world elections&lt;/a&gt; at &lt;a href=&#34;https://www.jstor.org/stable/2748722?seq=1#page_scan_tab_contents&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;lower rates&lt;/a&gt;. Our study provides evidence that probabilistic forecasts give people more confidence that one candidate will win and suggestive evidence that we should expect them to vote at lower rates after seeing probabilistic forecasts.&lt;/p&gt;
&lt;p&gt;This matters &lt;em&gt;a lot more&lt;/em&gt; if one candidate’s potential voters are differentially affected, and there’s evidence that may be the case.&lt;/p&gt;
&lt;p&gt;1. Figure 2C in Study 1 suggests that the &lt;em&gt;candidate who is ahead&lt;/em&gt; in the polls will be more affected by the increased certainty that probabilistic forecasts convey.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/certaintyc.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;2. When you look at the balance of coverage of probabilistic forecasts on major television broadcasts, there is more coverage on MSNBC, which has a more liberal audience.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/msnbc_mentions.png&#34; alt=&#34;half&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;3. Consider who shares this material in social media&amp;ndash;specifically the average self-reported ideology of people who share links to various sites hosting poll-aggregators on Facebook, data that come from &lt;a href=&#34;http://science.sciencemag.org/content/early/2015/05/06/science.aaa1160.full&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this paper&lt;/a&gt;’s &lt;a href=&#34;http://dx.doi.org/10.7910/DVN/LDJ7MS&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;replication materials&lt;/a&gt;. The websites that present their results in terms of probabilities have left-leaning (negative) social media audiences. Only realclearpolitics.com, which doesn’t emphasize win-probabilities, has a conservative audience:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/bma_science_alignment.png&#34; alt=&#34;half&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;4. In 2016, the proportion of American National Election Study (ANES) respondents who thought the leading candidate would “win by quite a bit” was unusually high for Democrats&amp;hellip;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/anes_turnout_closerace_mc_tall.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;5. And we know that people who say the leading presidential candidate will “win by quite a bit” in pre-election polling are about three percentage points less likely to report voting shortly after the election than people who say it’s a close race — and that’s after conditioning on election year, prior turnout, and party identification. The data here are from the ANES and go back to 1952.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/closerace_vote_anes.png&#34; alt=&#34;normal&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;These data do not conclusively show that probabilistic forecasts affected turnout in the 2016 election, but they do raise questions about the real world consequences of probabilistic forecasts.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What about media narratives?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;We acknowledge that these effects may change depending on the context in which people encounter them — though people can certainly encounter a lone probability number in media coverage of probabilistic forecasts. We also acknowledge that our work cannot address how these effects compare to and/or interact with media narratives.&lt;/p&gt;
&lt;p&gt;However, other work that is relevant to this question has found that aggregating all polls &lt;a href=&#34;https://academic.oup.com/poq/article-abstract/80/4/943/2738970?redirectedFrom=fulltext&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;reduces the likelihood that news outlets&lt;/a&gt; focus on unusual polls that are more sensational or support a particular narrative.&lt;/p&gt;
&lt;p&gt;In some ways, the widespread success and reliance on these forecasts represents a triumph of scientific communication. In addition to greater precision compared with one-off horserace polls, probabilistic forecasts can quantify how likely a given U.S. presidential candidate is to win using polling data and complex simulation, rather than leaving the task of making sense of state and national polls to speculative commentary about “paths to victory,” as we point out in the &lt;a href=&#34;https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3117054&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paper&lt;/a&gt;. And as one of the hosts noted, we aren’t calling for an end to election projections.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Future work&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;We agree with the hosts that there are open questions about whether the public gives more weight to these probabilistic forecasts than other polling results and speculative commentary. We have also heard questions raised about how much probabilistic forecasts might &lt;em&gt;drive&lt;/em&gt; media narratives. These questions may prove difficult to answer and we encourage research that explores them.&lt;/p&gt;
&lt;p&gt;We hope this research continues to create a dialogue about how to best communicate polling data to the public. We would love to see more research into how the public consumes and is affected by election projections, including finding the most effective ways to convey uncertainty.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Know your data - Pricing diamonds using scatterplots and predictive models</title>
      <link>https://solmessing.netlify.app/post/visualization-series-scatterplot-understanding-the-diamond-market/</link>
      <pubDate>Sun, 02 Feb 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/visualization-series-scatterplot-understanding-the-diamond-market/</guid>
      <description>&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/fbfa61f67413c8e2805c507a14b38c24c5373265.png&#34; alt=&#34;ggpairs&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;My last post railed against the &lt;a href=&#34;https://solmessing.netlify.app/post/visualization-series-insight-from-cleveland-and-tufte-on-plotting-numeric-data-by-groups/&#34;&gt;bad visualizations that people often use to plot quantitive data by groups, and pitted pie charts, bar charts and dot plots against each other for two visualization tasks.  Dot plots came out on top&lt;/a&gt;.  I argued that this is because humans are good at the cognitive task of comparing position along a common scale, compared to making judgements about length, area, shading, direction, angle, volume, curvature, etc.&amp;mdash;a finding credited to Cleveland and McGill.  I enjoyed writing it and people seemed to like it, so I&amp;rsquo;m continuing my visualization series with the scatterplot.&lt;/p&gt;
&lt;h2 id=&#34;scatterplots&#34;&gt;Scatterplots&lt;/h2&gt;
&lt;p&gt;A scatterplot is a two-dimensional plane on which we record the intersection of two measurements for a set of case items&amp;ndash;usually two quantitative variables.  Just as humans are good at comparing position along a common scale in one dimension, our visual capabilities allow us to make fast, accurate judgements and recognize patterns when presented with a series of dots in two dimensions. This makes the scatterplot a valuable tool for data analysts both when exploring data and when communicating results to others.&lt;/p&gt;
&lt;p&gt;In this post&amp;mdash;part 1&amp;mdash;I&amp;rsquo;ll demonstrate various uses for scatterplots and outline some strategies to help make sure key patterns are not obscured by the scale or qualitative group-level differences in the data (e.g., the relationship between test scores and income differs for men and women). The motivation in this post is to come up with a model of diamond prices that you can use to help make sure you don&amp;rsquo;t get ripped off, specified based on insight from exploratory scatterplots combined with (somewhat) informed speculation. In part 2, I&amp;rsquo;ll discuss the use of panels aka facets aka small multiples to shed additional light on key patterns in the data, and local regression (loess) to examine central tendencies in the data. There are far fewer bad examples of this kind of visualization in the wild than the 3D barplots and pie charts mocked in my last post, though I was still able to find &lt;a href=&#34;http://www.showmethemath.com/Concepts_Explained/Scatter_Plot/homeworkScatterPlotAnswer.gif&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this lovely scatterplot + trend-line&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/0e5cd98eb90fbc27e55e776f3303057ef7a35dcb.gif&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h2 id=&#34;scatterplots-and-the-cartesian-coordinate-system&#34;&gt;Scatterplots and the Cartesian coordinate system&lt;/h2&gt;
&lt;p&gt;The scatterplot has a richer history than the visualizations I wrote about in my last post.  The scatterplot&amp;rsquo;s face forms a two-dimensional Cartesian coordinate system, and DeCartes&amp;rsquo; invention/discovery of this eponymous plane in around 1657 represents one of the most fundamental developments in science.  The Cartesian plane unites measurement, algebra, and geometry, depicting the relationship between variables (or functions) visually. Prior to the Cartesian plane, mathematics was divided into algebra and geometry, and the unification of the two made many new developments possible.  Of course, this includes modern map-making&amp;mdash;cartography, but the &lt;a href=&#34;http://en.wikipedia.org/wiki/Cartesian_coordinate_system#History&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Cartesian plane was also an important step in the development of calculus&lt;/a&gt;, without which very little of our modern would would be possible.&lt;/p&gt;
&lt;p&gt;The scatterplot is a powerful tool to help understand the relationship between variables, and especially if that relationship is non-linear. Say you want to get a sense of whether you&amp;rsquo;re paying the right price when shopping for a diamond. You can use data on the price and characteristics of many diamonds to help figure out whether the price advertised for any given diamond is reasonable, and you can use scatterplots to help figure out how to model that data in a sensible way. Consider the important relationship between the price of a diamond and its carat weight (which corresponds to its size):&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/635214c79e184de850272a4790deed0dc870a49a.png&#34; alt=&#34;caratprice&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;A few things pop out right away. We can see a non-linear relationship, and we can also see that the dispersion (variance) of the relationship also increases as carat size increases. With just a quick look at a scatterplot of the data, we&amp;rsquo;ve learned two important things about the functional relationship between price and carat size. And, we also therefore learned that running a linear model on this data as-is would be a bad idea.&lt;/p&gt;
&lt;h2 id=&#34;diamonds&#34;&gt;Diamonds&lt;/h2&gt;
&lt;p&gt;&lt;a name=&#34;Diamonds&#34;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;If you&amp;rsquo;ve ever used R, you&amp;rsquo;ve probably seen references to the diamonds data set that ships with Hadley Wickham&amp;rsquo;s ggplot2. It records the carat size and the price of more than 50 thousand diamonds, from &lt;a href=&#34;http://www.diamondse.info/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;http://www.diamondse.info/&lt;/a&gt; collected in &lt;a href=&#34;http://r.789695.n4.nabble.com/Year-of-data-collection-for-diamonds-dataset-in-ggplot2-td4506598.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;in 2008&lt;/a&gt;, and if you&amp;rsquo;re in the market for a diamond, exploring this data set can help you understand what&amp;rsquo;s in store and at what price point. This is particularly useful because each diamond is unique in a way that isn&amp;rsquo;t true of most manufactured products we are used to buying&amp;mdash;you can&amp;rsquo;t just plug a model number and look up the price on Amazon. And even an expert cannot cannot incorporate as much information about price as a picture of the entire market informed by data (though there&amp;rsquo;s no substitute for qualitative expertise to make sure your diamond is what the retailer claims).&lt;/p&gt;
&lt;p&gt;But even if you&amp;rsquo;re not looking to buy a diamond, the socioeconomic and political history of the diamond industry is fascinating. Diamonds birthed the mining industry in South Africa, which is now by far the largest and most advanced economy in Africa. I worked a summer in Johannesburg, and can assure you that South Africa&amp;rsquo;s cities look far more like L.A. and San Francisco than Lagos, Cairo, Mogadishu, Nairobi, or Rabat. Diamonds have stoked conflicts ranging from the Boer Wars to modern day wars in Sierra Leone, Liberia, Côte d&amp;rsquo;Ivoire, Zimbabwe and the DRC, where the 200 carat Millennium Star diamond was sold to DeBeers at the height of the civil war in the 1990s. Diamonds were one of the few assets that Jews could conceal from the Nazis during &lt;a href=&#34;http://www.archives.gov/research/holocaust/articles-and-papers/turning-history-into-justice.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;the &amp;ldquo;Aryanization of Jewish property&amp;rdquo;&lt;/a&gt; in the 1930s, and the Congressional Research Service reports that &lt;a href=&#34;http://royce.house.gov/uploadedfiles/rl30751.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Al Qaeda has used conflict diamonds to skirt international sanctions and finance operations from the 1998 East Africa Bombings to the September 11th attacks&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/c7c2fd41f3cf8c8b7423ab84c12dfbd14fed71ca.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Though the diamonds data set is full of prices and fairly esoteric certification ratings, hidden in the data are reflections of how a &lt;a href=&#34;http://www.nytimes.com/2013/05/05/fashion/weddings/how-americans-learned-to-love-diamonds.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;legendary marketing campaign permeated and was subsumed by our culture&lt;/a&gt;, hints about how different social strata responded, and insight into how the diamond market functions as a result.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;http://www.theatlantic.com/magazine/archive/1982/02/have-you-ever-tried-to-sell-a-diamond/304575/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;The story starts in 1870&lt;/a&gt; according to The Atlantic, when many tons of diamonds were discovered in South Africa near the Orange River.  Until then, diamonds were rare&amp;mdash;only a few pounds were mined from India and Brazil each year.  At the time diamonds had no use outside of jewelry as they do today in many industrial applications, so price depended only on scarce supply.  Hence, the project&amp;rsquo;s investors formed the De Beers Cartel in 1888 to control the global price&amp;mdash;by most accounts the most successful cartel in history, &lt;a href=&#34;http://en.wikipedia.org/wiki/De_Beers#Diamond_monopoly&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;controlling 90% of the world&amp;rsquo;s diamond supply until about 2000&lt;/a&gt;.  But World War I and the Great Depression saw diamond sales plummet.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/0346d018cd97813448c1ce59a20353ed7c900013.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;In 1938, according to the New York Times&amp;rsquo; account, the De Beers cartel wrote Philadelphia ad agency N. W. Ayer &amp;amp; Son, to investigate whether &amp;ldquo;the use of propaganda in various forms&amp;rdquo; might jump-start diamond sales in the U.S., which looked like the only potentially viable market at the time.  Surveys showed diamonds were low on the list of priorities among most couples contemplating marriage&amp;mdash;a luxury for the rich, &amp;ldquo;money down the drain.&amp;rdquo;  Frances Gerety, who the Times compares to Madmen&amp;rsquo;s Peggy Olson, took on the DeBeers&amp;rsquo; account at N.W. Ayer &amp;amp; Son, and worked toward the company&amp;rsquo;s goal &amp;ldquo;to create a situation where almost every person pledging marriage feels compelled to acquire a diamond engagement ring.&amp;rdquo;  A few years later, she coined the slogan, &amp;ldquo;Diamonds are forever.&amp;rdquo;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/90a8efdb27b9c59fbf6566591c14165beab4bfd6.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;The Atlantic&amp;rsquo;s Jay Epstein argues that this campaign gave birth to modern demand-advertising&amp;mdash;the objective was not direct sales, nor brand strengthening, but simply to impress the glamour, sentiment and emotional charge contained in the product itself.  The company gave diamonds to movie stars, sent out press packages emphasizing the size of diamonds celebrities gave each other, loaned diamonds to socialites attending prominent events like the Academy Awards and Kentucky Derby, and persuaded the British royal family to wear diamonds over other gems.  The diamond was also marketed as a status symbol, to reflect &amp;ldquo;a man&amp;rsquo;s &amp;hellip; success in life,&amp;rdquo; in ads with &amp;ldquo;the aroma of tweed, old leather and polished wood which is characteristic of a good club.&amp;rdquo;  A 1980s ad introduced the two-month benchmark: &amp;ldquo;Isn&amp;rsquo;t two months&amp;rsquo; salary a small price to pay for something that lasts forever?&amp;rdquo;&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/4f1b0bd62c2ab1b46e453b707d2981d84879d4ed.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;By any reasonable measure, Frances Gerety succeeded&amp;mdash;getting engaged means getting a diamond ring in America. Can you think of a movie where two people get engaged without a diamond ring? When you announce your engagement on Facebook, what icon does the site display?  Still think this marketing campaign might not be the most successful mass-persuasion effort in history?  I present to you a James Bond film, whose title bears the diamond cartel&amp;rsquo;s trademark:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/f4eda42fe398a38837e01e97e4d07606a20171fe.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Awe-inspiring and terrifying.  Let&amp;rsquo;s open the data set.  &lt;/p&gt;
&lt;p&gt;The first thing you should consider doing is plotting key variables against each other using the ggpairs() function.  This function plots every variable against every other, pairwise.  For a data set with as many rows as the diamonds data, you may want to sample first otherwise things will take a long time to render.  Also, if your data set has more than about ten columns, there will be too many plotting windows, so subset on columns first.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Uncomment these lines and install if necessary:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;GGally&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;ggplot2&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;scales&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;memisc&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ggplot2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;GGally&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;scales&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;diasamp&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;[sample&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;length&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;10000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggpairs&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diasamp&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;params&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;shape&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;I&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;.&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;outlier.shape&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;I&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;.&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;!-- * R style note: I started using the &#34;=&#34; operator over &#34;&lt;-&#34; after reading [John Mount&#39;s post on the topic](http://www.win-vector.com/blog/2013/04/prefer-for-assignment-in-r/?utm_source=rss&amp;utm_medium=rss&amp;utm_campaign=prefer-for-assignment-in-r), which shows how using &#34;&lt;-&#34; (but not &#34;=&#34;) incorrectly can result in silent errors.  There are other good reasons: 1.) WordPress and R-Bloggers occasionally mangle &#34;&lt;-&#34; thinking it is HTML code in ways unpredictable to me; 2.) &#34;=&#34; is what every other programming language uses; and 3.) (as pointed out by Alex Foss in comments) consider &#34;foo&lt;-3&#34; --- did the author mean to assign foo to 3 or to compare foo to -3?  Plus, 4.) the way R interprets that expression depends on white space---and if I&#39;m using an editor like Emacs or Sublime where I don&#39;t have a shortcut key assigned to &#34;&lt;-&#34;, I sometimes get the whitespace wrong.  This means spending extra time and brainpower on debugging, both of which are in short supply.  
 --&gt;
&lt;p&gt;Anyway, here&amp;rsquo;s the plot:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/5226c40443deaf7c082cd464531f4e27c0f151be.png&#34; alt=&#34;ggpairs&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;What&amp;rsquo;s happening is that ggpairs is plotting each variable against the other in a pretty smart way. In the lower-triangle of plot matrix, it uses grouped histograms for qualitative-qualitative pairs and scatterplots for quantitative-quantitative pairs.  In the upper-triangle, it plots grouped histograms for qualitative-qualitative pairs (using the x-instead of y-variable as the grouping factor), boxplots for qualitative-quantitative pairs, and provides the correlation for quantitative-quantitative pairs.
What we really care about here is price, so let&amp;rsquo;s focus on that.  We can see what might be relationships between price and clarity, and color, which we&amp;rsquo;ll keep in mind for later when we start modeling our data, but the critical factor driving price is the size/weight of a diamond. Yet as we saw above, the relationship between price and diamond size is non-linear. What might explain this pattern?  On the supply side, larger contiguous chunks of diamonds without significant flaws are probably much harder to find than smaller ones.  This may help explain the exponential-looking curve&amp;mdash;and I thought I noticed this when I was shopping for a diamond for my soon-to-be wife. Of course, this is related to the fact that the weight of a diamond is a function of volume, and volume is a function of x * y * z, suggesting that we might be especially interested in the cubed-root of carat weight.&lt;/p&gt;
&lt;p&gt;On the demand side, customers in the market for a less expensive, smaller diamond are probably more sensitive to price than more well-to-do buyers. Many less-than-one-carat customers would surely never buy a diamond were it not for the social norm of presenting one when proposing.  And, there are &lt;em&gt;fewer&lt;/em&gt; consumers who can afford a diamond larger than one carat.  Hence, we shouldn&amp;rsquo;t expect the market for bigger diamonds to be as competitive as that for smaller ones, so it makes sense that the variance as well as the price would increase with carat size.&lt;/p&gt;
&lt;p&gt;Often the distribution of any monetary variable will be highly skewed and vary over orders of magnitude. This can result from path-dependence (e.g., the rich get richer) and/or the multiplicitive processes (e.g., year on year inflation) that produce the ultimate price/dollar amount. Hence, it&amp;rsquo;s a good idea to look into compressing any such variable by putting it on a log scale (for more take a look at &lt;a href=&#34;http://www.r-statistics.com/2013/05/log-transformations-for-skewed-and-wide-distributions-from-practical-data-science-with-r/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this guest post on Tal Galili&amp;rsquo;s blog&lt;/a&gt;).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;binwidth&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;100&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;binwidth&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_x_log10&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price (log10)&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/0cc713fc225f620a8284b05a00900ec51e379221.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/cdd7c96b96c0c2deef8844c214703a2243a3070b.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Indeed, we can see that the prices for diamonds are heavily skewed, but when put on a log10 scale seem much better behaved (i.e., closer to the bell curve of a normal distribution).  In fact, we can see that the data show some evidence of bimodality on the log10 scale, consistent with our two-class, &amp;ldquo;rich-buyer, poor-buyer&amp;rdquo; speculation about the nature of customers for diamonds.
Let&amp;rsquo;s re-plot our data, but now let&amp;rsquo;s put price on a log10 scale:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_y_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log10_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price (log10) by Carat&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/f7fcbbe09988fd9cddc474aecbbd918c448b7575.png&#34; alt=&#34;caratpricelog10&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Better, though still a little funky&amp;mdash;let&amp;rsquo;s try using use the cube-root of carat as we speculated about above:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;cubroot_trans&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;function&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;trans_new&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;cubroot&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;transform&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;function&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;^&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;inverse&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;function&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x^3&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_x_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;cubroot_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_y_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log10_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;5000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;10000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price (log10) by Cubed-Root of Carat&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/271f4d38318b2eefdbe9c5ca6ed34a467e1dab66.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Nice, looks like an almost-linear relationship after applying the transformations above to get our variables on a nice scale.&lt;/p&gt;
&lt;h2 id=&#34;overplotting&#34;&gt;Overplotting&lt;/h2&gt;
&lt;hr&gt;
&lt;p&gt;Note that until now I haven&amp;rsquo;t done anything about overplotting&amp;mdash;where multiple points take on the same value, often due to rounding.  Indeed, price is rounded to dollars and carats are rounded to two digits.  Not bad, though when we&amp;rsquo;ve got this much data we&amp;rsquo;re going to have some serious overplotting.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;head&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;sort&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;table&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;decreasing&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;kc&#34;&gt;TRUE&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;head&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;sort&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;table&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;decreasing&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;kc&#34;&gt;TRUE&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;pre&gt;&lt;code&gt; 0.3 0.31 1.01  0.7 0.32    1 
2604 2249 2242 1981 1840 1558 

605 802 625 828 776 698 
132 127 126 125 124 121 
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Often you can deal with this by making your points smaller, using &amp;ldquo;jittering&amp;rdquo; to randomly shift points to make multiple points visible, and using transparency, which can be done in ggplot using the &amp;ldquo;alpha&amp;rdquo; parameter.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;ggplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_point&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alpha&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.75&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;position&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;jitter&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_x_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;cubroot_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_y_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log10_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;5000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;10000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price (log10) by Cubed-Root of Carat&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/8f93ee4c395a7382a2893b01d640b65e7006ebe9.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;This gives us a better sense of how dense and sparse our data is at key places.&lt;/p&gt;
&lt;h2 id=&#34;using-color-to-understand-qualitative-factors&#34;&gt;Using Color to Understand Qualitative Factors&lt;/h2&gt;
&lt;hr&gt;
&lt;p&gt;When I was looking around at diamonds, I also noticed that clarity seemed to factor in to price.  Of course, many consumers are looking for a diamond of a certain size, so we shouldn&amp;rsquo;t expect clarity to be as strong a factor as carat weight. And I must admit that even though my grandparents were jewelers, I initially had a hard time discerning a diamond rated VVS1 from one rated SI2. Surely most people need a loop to tell the difference. And, &lt;a href=&#34;http://www.bluenile.com/diamonds/diamond-cut&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;according to BlueNile, the cut of a diamond has a much more consequential impact on that &amp;ldquo;fiery&amp;rdquo; quality that jewelers describe as the quintessential characteristic of a diamond&lt;/a&gt;.  On clarity, the website states, &amp;ldquo;&lt;a href=&#34;http://www.bluenile.com/diamonds/diamond-clarity&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Many of these imperfections are microscopic, and do not affect a diamond&amp;rsquo;s beauty in any discernible way&lt;/a&gt;.&amp;rdquo; Yet, clarity seems to explain an awful lot of the remaining variance in price when we visualize it as a color on our plot:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;ggplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;colour&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;clarity&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_point&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alpha&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.75&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;position&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;jitter&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_colour_brewer&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;type&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;div&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;guide&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;guide_legend&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;title&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;reverse&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;override.aes&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;list&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alpha&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_x_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;cubroot_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_y_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log10_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;5000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;10000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;theme&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;legend.key&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;element_blank&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;())&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price (log10) by Cubed-Root of Carat and Color&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/8ec0d95374a095c4268c9cfad923354f819bbdfb.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Despite what BlueNile says, we don&amp;rsquo;t see as much variation on cut (though most diamonds in this data set are ideal cut anyway):&lt;/p&gt;
&lt;hr&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;ggplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;colour&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cut&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_point&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alpha&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.75&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;position&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;jitter&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_colour_brewer&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;type&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;div&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;guide&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;guide_legend&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;title&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;reverse&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;override.aes&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;list&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alpha&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_x_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;cubroot_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_y_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log10_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;5000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;10000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;theme&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;legend.key&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;element_blank&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;())&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price (log10) by Cube-Root of Carat and Cut&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/1ef4c90edca2b6e2014d25f2a901a221f9926f6e.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Color seems to explain some of the variance in price as well, though &lt;a href=&#34;http://www.bluenile.com/diamonds/diamond-color&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;BlueNile states that all color grades from D-J are basically not noticeable&lt;/a&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;ggplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamonds&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;colour&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_point&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alpha&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.75&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;position&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;jitter&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_colour_brewer&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;type&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;div&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;guide&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;guide_legend&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;title&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;reverse&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;override.aes&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;list&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alpha&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_x_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;cubroot_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_y_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;trans&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log10_trans&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;limits&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;breaks&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;350&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;5000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;10000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;15000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;theme&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;legend.key&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;element_blank&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;())&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Price (log10) by Cube-Root of Carat and Color&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/277ef617f6ebe3749e64a855be635406ea80dba7.png&#34; alt=&#34;caratpricecolorlog10&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;At this point, we&amp;rsquo;ve got a pretty good idea of how we might model price. But there are a few problems with our 2008 data&amp;mdash;not only do we need to account for inflation but the diamond market is quite different now than it was in 2008. In fact, when I fit models to this data then attempted to predict the price of diamonds I found on the market, I kept getting predictions that were far too low. After some additional digging, I found the &lt;a href=&#34;http://www.bain.com/publications/articles/global-diamond-report-2013.aspx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Global Diamond Report&lt;/a&gt;. It turns out that prices plummeted in 2008 due to the global financial crisis, and since then prices (at least for wholesale polished diamond) have grown at a roughly a 6 percent compound annual rate. The &lt;a href=&#34;http://diamonds.blogs.com/diamonds_update/diamond-prices/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rapidly-growing number of couples in China buying diamond engagement rings&lt;/a&gt; might also help explain this increase. After looking at data on PriceScope, I realized that &lt;a href=&#34;http://www.pricescope.com/diamond-prices/diamond-prices-chart&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;diamond prices grew unevenly across different carat sizes&lt;/a&gt;, meaning that the model I initially estimated couldn&amp;rsquo;t simply be adjusted by inflation. While I could have done ok with that model, I really wanted to estimate a new model based on fresh data.&lt;/p&gt;
&lt;p&gt;Thankfully I was able to put together a &lt;a href=&#34;https://github.com/solomonm/diamonds-data/blob/master/dinfo.py&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;python script to scrape diamondse.info&lt;/a&gt; without too much trouble. This dataset is about 10 times the size of the 2008 diamonds data set and features diamonds from all over the world certified by an array of authorities besides just the Gemological Institute of America (GIA). You can read in this data as follows (be forewarned&amp;mdash;it&amp;rsquo;s over 500K rows):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;RCurl&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;RCurl&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;diamondsurl&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;getBinaryURL&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;https://raw.github.com/solomonm/diamonds-data/master/BigDiamonds.Rda&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;load&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;rawConnection&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamondsurl&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;My &lt;a href=&#34;https://github.com/solomonm/diamonds-data&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;github repository has the code necessary to replicate each of the figures above&lt;/a&gt;&amp;mdash;most look quite similar, though this data set contains much more expensive diamonds than the original. Regardless of whether you&amp;rsquo;re using the original diamonds data set or the current larger diamonds data set, you can estimate a model based on what we learned from our scatterplots. We&amp;rsquo;ll regress carat, the cubed-root of carat, clarity, cut and color on log-price. I&amp;rsquo;m using only GIA-certified diamonds in this model and looking only at diamonds under $10K because these are the type of diamonds sold at most retailers I&amp;rsquo;ve seen and hence the kind I care most about. By trimming the most expensive diamonds from the dataset, our model will also be less likely to be thrown off by outliers at the high end of price and carat. The new data set has mostly the same columns as the old one, so we can just run the following (if you want to run it on the old data set, just set data=diamonds).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;diamondsbig&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;logprice&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;log&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamondsbig&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;price&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;m1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;logprice&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;I&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;^&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;diamondsbig[diamondsbig&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;price&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;10000&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;amp;&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;diamondsbig&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cert&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;==&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;GIA&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;m2&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;update&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;. &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;m3&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;update&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;. &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;cut&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;m4&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;update&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;. &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;clarity&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;memisc&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;memisc&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;mtable&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;m2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;m3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;m4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Here are the results for my recently scraped data set:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;===============================================================
                    m1          m2          m3          m4     
---------------------------------------------------------------
(Intercept)       2.671***    1.333***    0.949***   -0.464*** 
                 (0.003)     (0.012)     (0.012)     (0.009)   
I(carat^(1/3))    5.839***    8.243***    8.633***    8.320*** 
                 (0.004)     (0.022)     (0.021)     (0.012)   
carat                        -1.061***   -1.223***   -0.763*** 
                             (0.009)     (0.009)     (0.005)   
cut: V.Good                               0.120***    0.071*** 
                                         (0.002)     (0.001)   
cut: Ideal                                0.211***    0.131*** 
                                         (0.002)     (0.001)   
color: K/L                                            0.117*** 
                                                     (0.003)   
color: J/L                                            0.318*** 
                                                     (0.002)   
color: I/L                                            0.469*** 
                                                     (0.002)   
color: H/L                                            0.602*** 
                                                     (0.002)   
color: G/L                                            0.665*** 
                                                     (0.002)   
color: F/L                                            0.723*** 
                                                     (0.002)   
color: E/L                                            0.756*** 
                                                     (0.002)   
color: D/L                                            0.827*** 
                                                     (0.002)   
clarity: I1                                           0.301*** 
                                                     (0.006)   
clarity: SI2                                          0.607*** 
                                                     (0.006)   
clarity: SI1                                          0.727*** 
                                                     (0.006)   
clarity: VS2                                          0.836*** 
                                                     (0.006)   
clarity: VS1                                          0.891*** 
                                                     (0.006)   
clarity: VVS2                                         0.935*** 
                                                     (0.006)   
clarity: VVS1                                         0.995*** 
                                                     (0.006)   
clarity: IF                                           1.052*** 
                                                     (0.006)   
---------------------------------------------------------------
R-squared             0.888       0.892      0.899        0.969
N                338946      338946     338946       338946    
===============================================================
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now those are some very nice R-squared values&amp;mdash;we are accounting for almost all of the variance in price with the 4Cs.  If we want to know what whether the price for a diamond is reasonable, we can now use this model and exponentiate the result (since we took the log of price).  We need to multiply the result by exp(sigma^2/2), because the our error is no longer zero in expectation:&lt;/p&gt;




$$ 
\begin{align*}
E(log(y) \mid \mathbf{X} = \mathbf{x}) &amp;= E(\mathbf{X}\beta + \epsilon)\\
  E(y \mid \mathbf{X} = \mathbf{x}) &amp;= E( exp( \mathbf{X}\beta + \epsilon ) )\\
  &amp;= E( exp( \mathbf{X}\beta ) \times exp( \epsilon ) ) \\
  &amp;= E( exp( \mathbf{X}\beta ) ) \times E( exp( \epsilon ) ) \\
  &amp;= exp(\mathbf{X}\hat\beta) \times exp( \frac{\hat\sigma^2}{2} ) 
\end{align*}
$$


&lt;p&gt;To dig further into that last step, have a look at the &lt;a href=&#34;http://en.wikipedia.org/wiki/Log-normal_distribution#Arithmetic_moments&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Wikipedia page on log-normal distributed variables&lt;/a&gt;.
Thanks to &lt;a href=&#34;https://sites.google.com/site/miguelgodinhomatos/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Miguel&lt;/a&gt; for catching this.
Let&amp;rsquo;s take a look at an example from Blue Nile. I&amp;rsquo;ll use the full model, m4.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Example from BlueNile&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Round 1.00 Very Good I VS1 $5,601&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;thisDiamond&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;data.frame&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;carat&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.00&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;cut&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;V.Good&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;I&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;clarity&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;VS1&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;modEst&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;predict&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;newdata&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;thisDiamond&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;interval&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;prediction&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;level&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.95&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;exp&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;modEst&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;exp&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;sigma^2&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The results yield an expected value for price given the characteristics of our diamond and the upper and lower bounds of a 95% CI&amp;mdash;note that because this is a linear model, predict() is just multiplying each model coefficient by each value in our data. Turns out that this diamond is a touch pricier than expected value under the full model, though it is by no means outside our 95% CI. BlueNile has by most accounts a better reputation than diamondse.info however, and reputation is worth a lot in a business that relies on easy-to-forge certificates and one in which the non-expert can be easily fooled.&lt;/p&gt;
&lt;p&gt;This illustrates an important point about generalizing a model from one data set to another. First, there may be important differences between data sets&amp;mdash;as I&amp;rsquo;ve speculated about above&amp;mdash;making the estimates systematically biased. Second, overfitting&amp;mdash;our model may be fitting noise present in data set. Even a model cross-validated against out-of-sample predictions can be over-fit to noise that results in differences between data sets. Of course, while this model may give you a sense of whether your diamond is a rip-off against diamondse.info diamonds, it&amp;rsquo;s not clear that diamondse.info should be regarded as a source of universal truth about whether the price of a diamond is reasonable. Nonetheless, to have the expected price at diamondse.info with a 95% interval is a lot more information than we had about the price we should be willing to pay for a diamond before we started this exercise.&lt;/p&gt;
&lt;p&gt;An important point&amp;mdash;even though we can predict diamondse.info prices almost perfectly based on a function of the 4c&amp;rsquo;s, one thing that you should NOT conclude from this exercise is that &lt;em&gt;where&lt;/em&gt; you buy your diamond is irrelevant, which apparently used to be conventional wisdom in some circles.  You will almost surely pay more if you &lt;a href=&#34;https://web.archive.org/web/20131231035425/http://www.businessweek.com/articles/2013-05-06/tiffany-vs-dot-costco-which-diamond-ring-is-better&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;buy the same diamond at Tiffany&amp;rsquo;s versus Costco&lt;/a&gt;. But &lt;a href=&#34;https://web.archive.org/web/20140217105722/http://www.costco.com:80/2.12-ctw-Round-Brilliant-Cut-Internally-Flawless,-D-Color-Diamond-%22Audrey%22-Platinum-Wedding-Set.product.100006730.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Costco sells some pricy diamonds&lt;/a&gt; as well. Regardless, you can use this kind of model to give you an indication of whether you&amp;rsquo;re overpaying.&lt;/p&gt;
&lt;p&gt;Of course, the value of a natural diamond is largely socially constructed. Like money, diamonds are only valuable because society says they are&amp;mdash;-there&amp;rsquo;s no obvious economic efficiencies to be gained or return on investment in a diamond, except perhaps in a very subjective sense concerning your relationship with your significant other. To get a sense for just how much value is socially constructed, you can compare the price of a natural diamond to a synthetic diamond, which thanks to recent technological developments are of comparable quality to a &amp;ldquo;natural&amp;rdquo; diamond. Of course, natural diamonds fetch a dramatically higher price.&lt;/p&gt;
&lt;p&gt;One last thing&amp;mdash;there are few guarantees in life, and I offer none here. Though what we have here seems pretty good, data and models are never infallible, and obviously you can still get taken (or be persuaded to pass on a great deal) based on this model. Always shop with a reputable dealer, and make sure her incentives are aligned against selling you an overpriced diamond or worse one that doesn&amp;rsquo;t match its certificate. There&amp;rsquo;s no substitute for establishing a personal connection and lasting business relationship with an established jeweler you can trust.&lt;/p&gt;
&lt;h2 id=&#34;one-final-consideration&#34;&gt;One Final Consideration&lt;/h2&gt;
&lt;hr&gt;
&lt;p&gt;Plotting your data can help you understand it and can yield key insights.  But even scatterplot visualizations can be deceptive if you&amp;rsquo;re not careful.  Consider another data set the comes with the alr3 package&amp;mdash;soil temperature data from Mitchell, Nebraska, collected by Kenneth G. Hubbard from 1976-1992, which I came across in Weisberg, S. (2005). &lt;em&gt;Applied Linear Regression&lt;/em&gt;, 3rd edition. New York: Wiley (from which I&amp;rsquo;ve shamelessly stolen this example).
Let&amp;rsquo;s plot the data, naively:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;alr3&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;alr3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Mitchell&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Month&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Temp&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Mitchell&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/f02902f327e42be253eb2e245e5876de3395060b.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Looks kinda like noise.  What&amp;rsquo;s the story here?
When all else fails, think about it.
What&amp;rsquo;s on the X axis?  Month.  What&amp;rsquo;s on the Y-axis?  Temperature.  Hmm, well there are seasons in Nebraska, so temperature should fluctuate every 12 months.&lt;/p&gt;
&lt;p&gt;But we&amp;rsquo;ve put more than 200 months in a pretty tight space.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s stretch it out and see how it looks:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/11fe76cda575e832918edc0bccf89bd911dc04ee.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Don&amp;rsquo;t make that mistake.&lt;/p&gt;
&lt;p&gt;That concludes part I of this series on scatterplots.  Part II will illustrate the advantages of using facets/panels/small multiples, and show how tools to fit trendlines including linear regression and local regression (loess) can help yield additional insight about your data.&lt;/p&gt;
&lt;p&gt;You can also learn more about &lt;a href=&#34;https://www.udacity.com/course/ud651&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;exploratory data analysis via this Udacity course taught by my colleagues Dean Eckles and Moira Burke, and Chris Saden&lt;/a&gt;, which will be coming out in the next few weeks.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Facebook Privacy-Protected Full URLs Data Set</title>
      <link>https://solmessing.netlify.app/publication/messing-2020/</link>
      <pubDate>Wed, 01 Jan 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/messing-2020/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;Largest ever social science data set released, protected using &lt;a href=&#34;https://arxiv.org/abs/2002.04049&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;action-level differential privacy&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;See also &lt;a href=&#34;socialscience.one/blog/update-social-science-one&#34;&gt;Social Science One&amp;rsquo;s annoucement&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Media coverage: &lt;a href=&#34;https://www.sciencemag.org/news/2020/02/researchers-finally-get-access-data-facebook-s-role-political-discourse&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Science&lt;/a&gt;, &lt;a href=&#34;https://www.nature.com/articles/d41586-019-02966-x&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Nature&lt;/a&gt;, &lt;a href=&#34;https://www.poynter.org/fact-checking/2019/what-can-researchers-find-among-the-32-million-urls-facebook-just-released-to-social-science-one/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Poynter&lt;/a&gt;, &lt;a href=&#34;https://www.wired.com/story/facebook-social-network-becomes-social-science-subject/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Wired&lt;/a&gt;, &lt;a href=&#34;https://www.ft.com/content/6133b90e-e23f-11e9-9743-db5a370481bc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Financial Times&lt;/a&gt;, &lt;a href=&#34;https://techcrunch.com/2018/07/11/facebook-independent-research-commission-social-science-one-will-share-a-petabyte-of-user-data/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tech Crunch&lt;/a&gt;,
&lt;a href=&#34;https://www.theverge.com/2018/7/11/17561144/facebook-election-research-database-link-proposal&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;The Verge&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Guidelines for Implementing and Auditing Differentially Private Systems</title>
      <link>https://solmessing.netlify.app/publication/kifer-2020-guidelines/</link>
      <pubDate>Wed, 01 Jan 2020 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/kifer-2020-guidelines/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Display Jupyter Notebooks with Academic</title>
      <link>https://solmessing.netlify.app/post/jupyter/</link>
      <pubDate>Tue, 05 Feb 2019 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/jupyter/</guid>
      <description>&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-python&#34; data-lang=&#34;python&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;kn&#34;&gt;from&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;IPython.core.display&lt;/span&gt; &lt;span class=&#34;kn&#34;&gt;import&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Image&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Image&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;https://www.python.org/static/community_logos/python-logo-master-v3-TM-flattened.png&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;./index_1_0.png&#34; alt=&#34;png&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-python&#34; data-lang=&#34;python&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;print&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Welcome to Academic!&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;pre&gt;&lt;code&gt;Welcome to Academic!
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&#34;install-python-and-jupyterlab&#34;&gt;Install Python and JupyterLab&lt;/h2&gt;
&lt;p&gt;&lt;a href=&#34;https://www.anaconda.com/distribution/#download-section&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Install Anaconda&lt;/a&gt; which includes Python 3 and JupyterLab.&lt;/p&gt;
&lt;p&gt;Alternatively, install JupyterLab with &lt;code&gt;pip3 install jupyterlab&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id=&#34;create-or-upload-a-jupyter-notebook&#34;&gt;Create or upload a Jupyter notebook&lt;/h2&gt;
&lt;p&gt;Run the following commands in your Terminal, substituting &lt;code&gt;&amp;lt;MY-WEBSITE-FOLDER&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;SHORT-POST-TITLE&amp;gt;&lt;/code&gt; with the file path to your Academic website folder and a short title for your blog post (use hyphens instead of spaces), respectively:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;mkdir -p &amp;lt;MY-WEBSITE-FOLDER&amp;gt;/content/post/&amp;lt;SHORT-POST-TITLE&amp;gt;/
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;cd&lt;/span&gt; &amp;lt;MY-WEBSITE-FOLDER&amp;gt;/content/post/&amp;lt;SHORT-POST-TITLE&amp;gt;/
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;jupyter lab index.ipynb
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The &lt;code&gt;jupyter&lt;/code&gt; command above will launch the JupyterLab editor, allowing us to add Academic metadata and write the content.&lt;/p&gt;
&lt;h2 id=&#34;edit-your-post-metadata&#34;&gt;Edit your post metadata&lt;/h2&gt;
&lt;p&gt;The first cell of your Jupter notebook will contain your post metadata (&lt;a href=&#34;https://sourcethemes.com/academic/docs/front-matter/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;front matter&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;In Jupter, choose &lt;em&gt;Markdown&lt;/em&gt; as the type of the first cell and wrap your Academic metadata in three dashes, indicating that it is YAML front matter:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-fallback&#34; data-lang=&#34;fallback&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;---
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;title: My post&amp;#39;s title
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;date: 2019-09-01
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;# Put any other Academic metadata here...
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;---
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Edit the metadata of your post, using the &lt;a href=&#34;https://sourcethemes.com/academic/docs/managing-content&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;documentation&lt;/a&gt; as a guide to the available options.&lt;/p&gt;
&lt;p&gt;To set a &lt;a href=&#34;https://sourcethemes.com/academic/docs/managing-content/#featured-image&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;featured image&lt;/a&gt;, place an image named &lt;code&gt;featured&lt;/code&gt; into your post&amp;rsquo;s folder.&lt;/p&gt;
&lt;p&gt;For other tips, such as using math, see the guide on &lt;a href=&#34;https://wowchemy.com/docs/content/writing-markdown-latex/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;writing content with Academic&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;convert-notebook-to-markdown&#34;&gt;Convert notebook to Markdown&lt;/h2&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;jupyter nbconvert index.ipynb --to markdown --NbConvertApp.output_files_dir&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;.
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;example&#34;&gt;Example&lt;/h2&gt;
&lt;p&gt;This post was created with Jupyter. The orginal files can be found at &lt;a href=&#34;https://github.com/gcushen/hugo-academic/tree/master/exampleSite/content/post/jupyter&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://github.com/gcushen/hugo-academic/tree/master/exampleSite/content/post/jupyter&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Slides</title>
      <link>https://solmessing.netlify.app/slides/example/</link>
      <pubDate>Tue, 05 Feb 2019 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/slides/example/</guid>
      <description>&lt;h1 id=&#34;create-slides-in-markdown-with-wowchemy&#34;&gt;Create slides in Markdown with Wowchemy&lt;/h1&gt;
&lt;p&gt;&lt;a href=&#34;https://wowchemy.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Wowchemy&lt;/a&gt; | &lt;a href=&#34;https://wowchemy.com/docs/content/slides/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Documentation&lt;/a&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&#34;features&#34;&gt;Features&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Efficiently write slides in Markdown&lt;/li&gt;
&lt;li&gt;3-in-1: Create, Present, and Publish your slides&lt;/li&gt;
&lt;li&gt;Supports speaker notes&lt;/li&gt;
&lt;li&gt;Mobile friendly slides&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&#34;controls&#34;&gt;Controls&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Next: &lt;code&gt;Right Arrow&lt;/code&gt; or &lt;code&gt;Space&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Previous: &lt;code&gt;Left Arrow&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Start: &lt;code&gt;Home&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Finish: &lt;code&gt;End&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Overview: &lt;code&gt;Esc&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Speaker notes: &lt;code&gt;S&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Fullscreen: &lt;code&gt;F&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Zoom: &lt;code&gt;Alt + Click&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://revealjs.com/pdf-export/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PDF Export&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&#34;code-highlighting&#34;&gt;Code Highlighting&lt;/h2&gt;
&lt;p&gt;Inline code: &lt;code&gt;variable&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;Code block:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-python&#34; data-lang=&#34;python&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;porridge&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;blueberry&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;if&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;porridge&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;==&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;blueberry&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nb&#34;&gt;print&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Eating...&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;h2 id=&#34;math&#34;&gt;Math&lt;/h2&gt;
&lt;p&gt;In-line math: $x + y = z$&lt;/p&gt;
&lt;p&gt;Block math:&lt;/p&gt;
&lt;p&gt;$$
f\left( x \right) = ;\frac{{2\left( {x + 4} \right)\left( {x - 4} \right)}}{{\left( {x + 4} \right)\left( {x + 1} \right)}}
$$&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&#34;fragments&#34;&gt;Fragments&lt;/h2&gt;
&lt;p&gt;Make content appear incrementally&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-fallback&#34; data-lang=&#34;fallback&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;{{% fragment %}} One {{% /fragment %}}
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;{{% fragment %}} **Two** {{% /fragment %}}
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;{{% fragment %}} Three {{% /fragment %}}
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Press &lt;code&gt;Space&lt;/code&gt; to play!&lt;/p&gt;
&lt;span class=&#34;fragment &#34; &gt;
  One
&lt;/span&gt;
&lt;span class=&#34;fragment &#34; &gt;
  &lt;strong&gt;Two&lt;/strong&gt;
&lt;/span&gt;
&lt;span class=&#34;fragment &#34; &gt;
  Three
&lt;/span&gt;
&lt;hr&gt;
&lt;p&gt;A fragment can accept two optional parameters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;class&lt;/code&gt;: use a custom style (requires definition in custom CSS)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;weight&lt;/code&gt;: sets the order in which a fragment appears&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&#34;speaker-notes&#34;&gt;Speaker Notes&lt;/h2&gt;
&lt;p&gt;Add speaker notes to your presentation&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-markdown&#34; data-lang=&#34;markdown&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;{{% speaker_note %}}
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;-&lt;/span&gt; Only the speaker can read these notes
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;-&lt;/span&gt; Press &lt;span class=&#34;sb&#34;&gt;`S`&lt;/span&gt; key to view
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  {{% /speaker_note %}}
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Press the &lt;code&gt;S&lt;/code&gt; key to view the speaker notes!&lt;/p&gt;
&lt;aside class=&#34;notes&#34;&gt;
  &lt;ul&gt;
&lt;li&gt;Only the speaker can read these notes&lt;/li&gt;
&lt;li&gt;Press &lt;code&gt;S&lt;/code&gt; key to view&lt;/li&gt;
&lt;/ul&gt;

&lt;/aside&gt;
&lt;hr&gt;
&lt;h2 id=&#34;themes&#34;&gt;Themes&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;black: Black background, white text, blue links (default)&lt;/li&gt;
&lt;li&gt;white: White background, black text, blue links&lt;/li&gt;
&lt;li&gt;league: Gray background, white text, blue links&lt;/li&gt;
&lt;li&gt;beige: Beige background, dark text, brown links&lt;/li&gt;
&lt;li&gt;sky: Blue background, thin dark text, blue links&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;ul&gt;
&lt;li&gt;night: Black background, thick white text, orange links&lt;/li&gt;
&lt;li&gt;serif: Cappuccino background, gray text, brown links&lt;/li&gt;
&lt;li&gt;simple: White background, black text, blue links&lt;/li&gt;
&lt;li&gt;solarized: Cream-colored background, dark green text, blue links&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;

&lt;section data-noprocess data-shortcode-slide
  
      
      data-background-image=&#34;/media/boards.jpg&#34;
  &gt;

&lt;h2 id=&#34;custom-slide&#34;&gt;Custom Slide&lt;/h2&gt;
&lt;p&gt;Customize the slide style and background&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-markdown&#34; data-lang=&#34;markdown&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;{{&lt;span class=&#34;p&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;slide&lt;/span&gt; &lt;span class=&#34;na&#34;&gt;background-image&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;/media/boards.jpg&amp;#34;&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;&amp;gt;&lt;/span&gt;}}
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;{{&lt;span class=&#34;p&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;slide&lt;/span&gt; &lt;span class=&#34;na&#34;&gt;background-color&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;#0000FF&amp;#34;&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;&amp;gt;&lt;/span&gt;}}
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;{{&lt;span class=&#34;p&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;slide&lt;/span&gt; &lt;span class=&#34;na&#34;&gt;class&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;my-style&amp;#34;&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;&amp;gt;&lt;/span&gt;}}
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;h2 id=&#34;custom-css-example&#34;&gt;Custom CSS Example&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s make headers navy colored.&lt;/p&gt;
&lt;p&gt;Create &lt;code&gt;assets/css/reveal_custom.css&lt;/code&gt; with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-css&#34; data-lang=&#34;css&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;nc&#34;&gt;reveal&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;section&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;h1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;nc&#34;&gt;reveal&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;section&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;h2&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;nc&#34;&gt;reveal&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;section&lt;/span&gt; &lt;span class=&#34;nt&#34;&gt;h3&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;k&#34;&gt;color&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;navy&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;h1 id=&#34;questions&#34;&gt;Questions?&lt;/h1&gt;
&lt;p&gt;&lt;a href=&#34;https://discord.gg/z8wNYzb&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Ask&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://wowchemy.com/docs/content/slides/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Documentation&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Projecting confidence: How the probabilistic horse race confuses and demobilizes the public</title>
      <link>https://solmessing.netlify.app/publication/wlm-2019-projecting/</link>
      <pubDate>Tue, 01 Jan 2019 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/wlm-2019-projecting/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;Supplimentary &lt;a href=&#34;https://solmessing.netlify.app/pdf/aggregatorSM.pdf&#34;&gt;materials&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Cited by &lt;a href=&#34;https://fivethirtyeight.com/features/politics-podcast-whats-so-wrong-with-nancy-pelosi/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;FiveThirthyEight&amp;rsquo;s Politics
Podcast&lt;/a&gt;
as influential in decision to change forecast presentation.&lt;/li&gt;
&lt;li&gt;Media coverage: &lt;a href=&#34;https://www.washingtonpost.com/news/politics/wp/2018/02/06/clintons-achilles-heel-in-2016-may-have-been-overconfidence/?utm_term=.b71fb84da00f&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Washington
Post&lt;/a&gt;,
&lt;a href=&#34;http://nymag.com/daily/intelligencer/2018/02/americans-dont-understand-election-probabilities.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;New York
Magazine&lt;/a&gt;,
&lt;a href=&#34;https://politicalwire.com/2018/02/06/election-forecasts-lower-voter-turnout/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Political
Wire&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Tumor formation of adult stem cell transplants in rodent arthritic joints</title>
      <link>https://solmessing.netlify.app/publication/chapelin-2019-tumor/</link>
      <pubDate>Tue, 01 Jan 2019 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/chapelin-2019-tumor/</guid>
      <description></description>
    </item>
    
    <item>
      <title>How to break regression</title>
      <link>https://solmessing.netlify.app/post/how-to-break-regression/</link>
      <pubDate>Wed, 13 Jun 2018 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/how-to-break-regression/</guid>
      <description>&lt;p&gt;Regression models are a cornerstone of modern social science. They’re at the heart of efforts to estimate causal relationships between variables in a multivariate environment and are the basic building blocks of many machine learning models. Yet social scientists can run into a lot of situations where regression models break.&lt;/p&gt;
&lt;p&gt;Famed social psychologist Richard Nisbett &lt;a href=&#34;https://www.edge.org/conversation/richard_nisbett-the-crusade-against-multiple-regression-analysis&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;recently argued&lt;/a&gt; that regression analysis is so misused and misunderstood that analyses based on multiple regression “are often somewhere between meaningless and quite damaging.” (He was mainly talking about cases in which researchers publish correlational results that are covered in the media as causal statements about the world.)&lt;/p&gt;
&lt;p&gt;Below, I’ll walk through some of the potential pitfalls you might encounter when you fire up your favorite &lt;a href=&#34;https://seanjtaylor.com/post/39573264781/the-statistics-software-signal&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;statistical software&lt;/a&gt; package and run regressions. Specifically, I’ll be using simulation in R as an educational tool to help you better understand the ways in which regressions can break.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Using simulations to unpack regression&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The idea of using R simulations to help understand regression models was inspired by Ben Ogorek’s &lt;a href=&#34;http://anythingbutrbitrary.blogspot.com/2016/01/how-to-create-confounders-with.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;post&lt;/a&gt; on regression confounders and collider bias.&lt;/p&gt;
&lt;p&gt;The great thing about using simulation in this way is that you control the world that generates your data. The code I’ll introduce below represents the true &lt;em&gt;data-generating process&lt;/em&gt;,since I’m using R’s random number generators to simulate the data. In real life, of course, we only have the data we observe, and we don’t really know how the data-generating process works unless we have a solid theory (like Newtonian physics or evolution) where the system of relevant variables and causal relationships is well understood and to which there is really no analogous phenomenon in social science.&lt;/p&gt;
&lt;p&gt;What I’ll do here is create a dataset based on two random standard normal variables by simulating them using the &lt;em&gt;rnorm()&lt;/em&gt; function, which draws random values from a normal distribution with mean 0 and standard deviation 1, unless you specify otherwise. I’ll create a functional relationship between y and x such that a 1 unit increase in x will be associated with a .4 unit increase in y.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# make the code reproducible by setting a random number seed&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;set.seed&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;100&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# When everything works:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.4&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;hist&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;hist&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;y&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Now estimate our model:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.0348&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.7013&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.0085&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.6212&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;3.1688&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            		&lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; 	&lt;span class=&#34;m&#34;&gt;0.003921&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.031039&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.126&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.899&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;           	&lt;span class=&#34;m&#34;&gt;0.413415&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.030129&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;13.722&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.9814&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.1587&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.1579&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;188.3&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2.2e-16&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Plot it&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ggplot2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;y&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;geom_smooth&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;method&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;lm&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;The Perfect Regression&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Notice that the model estimates the functional relationship between x and y that I simulated quite well. The plot looks like this:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://cdn-images-1.medium.com/max/1600/1*0zIR7Mtuak5DamPOgEiHog.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;What about omitted variables? Our machinery actually still works if there is another factor causing y, as long as it is &lt;em&gt;uncorrelated&lt;/em&gt; with x.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The dreaded omitted variable bias&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Omitted variable bias (OVB) is much feared, and judging by the top internet search results, not well understood. Some top sources say it occurs when “&lt;a href=&#34;http://carecon.org.uk/UWEcourse/OVbias.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;an important&lt;/a&gt;” variable is missing or when a variable that “&lt;a href=&#34;https://en.wikipedia.org/wiki/Omitted-variable_bias&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;is correlated&lt;/a&gt;” with both x and y is missing. I even found a university &lt;a href=&#34;http://www3.wabash.edu/econometrics/EconometricsBook/chap18.htm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;econometrics&lt;/a&gt; course that defined OVB this way.&lt;/p&gt;
&lt;p&gt;But neither of those definitions are quite right. OVB occurs when a variable that &lt;em&gt;causes&lt;/em&gt; y is missing from the model (and is correlated with x). Let’s call that variable w. Because w is in play when we consider the causal relationship between x and y, it’s often referred to as “endogenous” or a “confounding variable.”&lt;/p&gt;
&lt;p&gt;The example below first demonstrates that w, our confounding variable, will bias our results if we fail to include it in our model. The next two examples are essentially a re-telling of the &lt;a href=&#34;http://anythingbutrbitrary.blogspot.com/2016/01/how-to-create-confounders-with.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;post I mentioned above&lt;/a&gt; on collider bias, but emphasizing slightly different points.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;w&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.4&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.3&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;w&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;m1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;c1&#34;&gt;# Omitted variable bias&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.2190&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.7025&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.0314&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.7120&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;3.1158&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.01126&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03310&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.34&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.734&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;0.50179&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03049&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;16.46&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.046&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.2135&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.2127&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;270.9&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2.2e-16&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;There it is: classic omitted variable bias. We only observed x, and the influence of the omitted variable w was attributed to x in our model. If you re-rerun the regression with w in the model, you no longer get biased estimates.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;m2&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;c1&#34;&gt;# No omitted variable bias after conditioning on w&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.2748&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.6632&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.0001&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.6933&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;2.9664&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.02841&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03141&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.905&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.366&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;0.40627&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03132&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;12.973&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;0.32344&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03439&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;9.405&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.9927&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.3024&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.301&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;216.1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2.2e-16&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that the regression errors, also known as residuals, are correlated with w:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now, recall above that I wrote that it’s wrong to say that OVB occurs when our omitted variable is correlated with both x and y. And yet w, x and w and y are all correlated in this first example:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;cor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;m1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;residuals&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;[1]&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.2597859&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;So why can’t we just say that OVB occurs when our omitted variable is correlated with both x and y? As the next example will show, correlation isn’t enough — w needs to &lt;em&gt;cause&lt;/em&gt; both x and y. We can easily imagine a case in which we don’t have causality but we still see this kind of correlation — when x and y both cause w.&lt;/p&gt;
&lt;p&gt;Let’s make this a little more concrete. Suppose we care about the effect of news media consumption (x) on voter turnout (y). One factor that some researchers think may cause both news media consumption and turnout is political interest (w). If we only measure media consumption and voter turnout, political interest is likely to confound our estimates.&lt;/p&gt;
&lt;p&gt;But another school of thought from social psychology — along the lines of self-perception theory and &lt;a href=&#34;https://en.wikipedia.org/wiki/Cognitive_dissonance&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;cognitive dissonance&lt;/a&gt; — suggests that the causality could be reversed: Voting behavior might be mostly determined by other factors, and casting a ballot might prompt us to be &lt;em&gt;more&lt;/em&gt; interested in political developments in the future. Similarly, watching the news might prompt us to become &lt;em&gt;more&lt;/em&gt; interested in politics. Let’s suppose that second school of thought is right. If so, our simulated data will look like this:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Political interest increases after consuming media and participating, and, &lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# in this hypothetical world, does *not* increase media consuption or participation&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;political_interest_w&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.2&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.6&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;cormat&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;cor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;as.matrix&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;data.frame&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;political_interest_w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;round&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cormat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                     &lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;political_interest_w&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt;                 &lt;span class=&#34;m&#34;&gt;1.00&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;0.11&lt;/span&gt;                 &lt;span class=&#34;m&#34;&gt;0.70&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt;                     &lt;span class=&#34;m&#34;&gt;0.11&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;1.00&lt;/span&gt;                 &lt;span class=&#34;m&#34;&gt;0.46&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;political_interest_w&lt;/span&gt;                &lt;span class=&#34;m&#34;&gt;0.70&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;0.46&lt;/span&gt;                 &lt;span class=&#34;m&#34;&gt;1.00&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As you can see, all factors are again correlated with each other. But this time, if we &lt;em&gt;only&lt;/em&gt; include x (media consumption) and y (turnout) in the equation, we get the correct estimate:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-2.8460&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.6972&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.0076&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.6702&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;3.3925&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                    &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;         &lt;span class=&#34;m&#34;&gt;-0.01202&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03217&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;-0.374&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.708839&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.11719&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03321&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;3.529&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.000436&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.014&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.01233&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.01134&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;12.46&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.0004359&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;What makes defining omitted variable bias based on correlation so dangerous is that if we now include w (political interest), we will get a different kind of bias — what’s called &lt;a href=&#34;https://en.wikipedia.org/wiki/Collider_%28epidemiology%29&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;collider bias&lt;/a&gt; or &lt;a href=&#34;https://www.annualreviews.org/doi/10.1146/annurev-soc-071913-043455&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;endogenous selection bias&lt;/a&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;political_interest_w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;voter_turnout_y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;political_interest_w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-2.1569&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.5981&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.0129&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.5701&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;2.8356&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                      &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;           &lt;span class=&#34;m&#34;&gt;0.003155&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.027098&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.116&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.907&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;media_consumption_x&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;-0.437084&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.039102&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-11.178&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;political_interest_w&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.444571&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.021928&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;20.274&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.854&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.3007&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.2993&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;214.3&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2.2e-16&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;strong&gt;Simpson’s paradox&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Simpson’s paradox often occurs in social science (and medicine, too) when you pool data instead of conditioning it on group membership (i.e., adding it as a factor in your regression model).&lt;/p&gt;
&lt;p&gt;Suppose that, all other things being equal, consuming media causes a slight shift in policy preferences toward the left. But, on average, Republicans consume more news than non-Republicans. And we know that generally Republicans have much more right-leaning preferences.&lt;/p&gt;
&lt;p&gt;If we just measure media consumption and policy preferences without including Republicans in the model, we’ll actually estimate that the effect goes in the direction &lt;em&gt;opposite&lt;/em&gt; of the true causal effect.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Let&amp;#39;s say that 40% of people in this population are Republicans&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rbinom&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# And they consume more media&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.75&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Consuming more media causes a slight leftward shift in policy&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# preferences, and Republicans have more right-leaning preferences&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-.2&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# for easier plotting later&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;data.frame&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;factor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;non-republican&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#34;republican&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;[df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# If we don&amp;#39;t condition on being Republican, we&amp;#39;ll actually estimate&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# that the effect goes in the *opposite* direction&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.6108&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.9559&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.0198&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.9257&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;3.9537&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                  &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;        &lt;span class=&#34;m&#34;&gt;0.68923&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.04323&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;15.94&lt;/span&gt;  &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.15269&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03966&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;3.85&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.000126&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.317&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.01463&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.01365&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;14.82&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.0001257&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Naive plot&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;geom_smooth&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;method&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;lm&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;Naive estimate (Simpson&amp;#39;s Paradox)&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The estimate goes in the opposite direction of the true effect! Here’s what the plot looks like:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://cdn-images-1.medium.com/max/1600/1*2gxiWJN7ElkYAO-wuvakzA.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;To resolve this paradox, we need to add a factor in the model that indicates whether or not a respondent is a Republican. Adding that factor lets us estimate &lt;em&gt;separate&lt;/em&gt; slopes for Republicans and non-Republicans. Note that this is &lt;em&gt;not&lt;/em&gt; like estimating an interaction term, where two explanatory variables are multiplied together. It’s not that the slopes are &lt;em&gt;different&lt;/em&gt;, we just need to estimate separate ones for Republicans and non-Republicans.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Condition on being a Republican to get the right estimates&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.5518&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.6678&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.0186&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.6562&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;3.3009&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                  &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;        &lt;span class=&#34;m&#34;&gt;0.05335&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03904&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;1.366&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.172&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.13615&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03111&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;-4.376&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.34e-05&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt;         &lt;span class=&#34;m&#34;&gt;1.93049&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.06758&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;28.565&lt;/span&gt;  &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.9774&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.4581&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.457&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;421.4&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2.2e-16&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Conditioning on being Republican&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;qplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;policy_prefs&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;colour&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;republican&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;scale_color_manual&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;values&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;blue&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;red&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;geom_smooth&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;method&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;lm&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;Conditioning on being a Republican (Simpson&amp;#39;s Paradox)&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Here’s what the plot looks like:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://cdn-images-1.medium.com/max/1600/1*rwNGDFcsSSMiGsJoc6ZxsQ.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Correlated errors&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Another cardinal sin — and one that we should worry a lot about because it often arises from social desirability bias in survey responses — is the phenomenon of correlated errors. This example is inspired by &lt;a href=&#34;https://www.nowpublishers.com/article/Details/QJPS-6005&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Vavreck (2007).&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Here, self-reported turnout and media consumption are caused by a combination of social desirability bias and true turnout and true consumption, respectively:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1000&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# The &amp;#34;Truth&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;true_media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;true_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# social desirability bias&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;social_desirability&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rnorm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#what we actually observe from self reports:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;self_report_media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;true_media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;social_desirability&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;self_report_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;true_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;social_desirability&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Let’s compare the estimated effect sizes of the self-reported data and the “true” data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Self reports&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;self_report_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;self_report_media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;self_report_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;self_report_media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.9604&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.7766&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.0142&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.8465&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;4.1811&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                              &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;                    &lt;span class=&#34;m&#34;&gt;0.02020&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03951&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.511&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.609&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;self_report_media_consumption&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.54605&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.02716&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;20.102&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.248&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.2882&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.2875&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;404.1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2.2e-16&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# &amp;#34;Truth&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;true_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;true_media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;true_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;true_media_consumption&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.5814&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.6677&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.0077&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.6829&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;3.4799&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                       &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;             &lt;span class=&#34;m&#34;&gt;0.01372&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03217&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.426&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.670&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;true_media_consumption&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.01313&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03245&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.404&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.686&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.017&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.0001639&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;-0.000838&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1636&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;998&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.686&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The self-reported data is biased toward over-estimating the effect size, a very dangerous problem. How could we fix this? Well, one way is to actually measure social desirability and include it in the model:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summary&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;self_report_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;self_report_media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;social_desirability&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Call&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;lm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;formula&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;self_report_vote&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;~&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;self_report_media_consumption&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;social_desirability&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residuals&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;Min&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;Median&lt;/span&gt;      &lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Q&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;Max&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;-3.6042&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.6774&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;-0.0127&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.6899&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;3.4470&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Coefficients&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                              &lt;span class=&#34;n&#34;&gt;Estimate&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Std.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;t&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;value&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;Pr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;gt;|&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Intercept&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;                    &lt;span class=&#34;m&#34;&gt;0.01208&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03220&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.375&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.708&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;self_report_media_consumption&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.01220&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.03246&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;0.376&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.707&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;social_desirability&lt;/span&gt;            &lt;span class=&#34;m&#34;&gt;1.02245&lt;/span&gt;    &lt;span class=&#34;m&#34;&gt;0.04547&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;22.487&lt;/span&gt;   &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2e-16&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;---&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Signif.&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;codes&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;***&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.001&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;**&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.01&lt;/span&gt; ‘&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;’ &lt;span class=&#34;m&#34;&gt;0.05&lt;/span&gt; ‘&lt;span class=&#34;n&#34;&gt;.’&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0.1&lt;/span&gt; ‘ ’ &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Residual&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;standard&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;error&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1.017&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;degrees&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;freedom&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Multiple&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.5277&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;	&lt;span class=&#34;n&#34;&gt;Adjusted&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;R&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;squared&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;m&#34;&gt;0.5268&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;statistic&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;   &lt;span class=&#34;m&#34;&gt;557&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;997&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;DF&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;  &lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2.2e-16&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that this while most people think about social desirability as being a problem related to measurement error, it is essentially the same problem as omitted variable bias, as described above.&lt;/p&gt;
&lt;p&gt;It’s important to remember that omitted variable bias and correlated errors are just two potential problems with regression analysis. Regression models are also not immune to issues associated with low levels of &lt;a href=&#34;https://en.wikipedia.org/wiki/Power_%28statistics%29&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;statistical power&lt;/a&gt;, the failure to account for the influence of extreme values, and &lt;a href=&#34;https://en.wikipedia.org/wiki/Heteroscedasticity&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;heteroskedasticity&lt;/a&gt;, among others. But by simulating the data-generating process, researchers can get a good sense of some of the more common ways in which statistical models might depart from reality.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Replication of &#39;Bias in the Flesh&#39;</title>
      <link>https://solmessing.netlify.app/post/replication_of_bias_in_the_flesh/</link>
      <pubDate>Mon, 16 Oct 2017 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/replication_of_bias_in_the_flesh/</guid>
      <description>&lt;p&gt;This post presents a replication of &lt;a href=&#34;https://solmessing.netlify.app/pdf/HSVmetricsCampaignsDarknessPOQFINAL.pdf&#34;&gt;Messing et al&lt;/a&gt;. (2016, study 2), which showed that exposure to darker images of Barack Obama increased stereotype activation, as indicated by the tendency to finish incomplete word prompts---such as &amp;ldquo;W E L _ _ _ _&amp;rdquo;---in stereotype-consistent ways (&amp;ldquo;WELFARE&amp;rdquo;).&lt;/p&gt;
&lt;p&gt;Overall, the replication shows that darker images of even &lt;a href=&#34;http://www.sciencedirect.com/science/article/pii/S0022103110002635&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;counter-stereotypical exemplars like Barack Obama&lt;/a&gt; can increase stereotype activation, but that the strength of the effect is weaker than conveyed in the original study.  A reanalysis of the original study conducted in the course of this replication effort unearthed a number of problems that, when corrected, yield estimates of the effect that are consistent with those documented in the replication. This reanalysis also follows.&lt;/p&gt;
&lt;p&gt;I&#39;m posting this to&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;disseminate a &lt;a href=&#34;https://solmessing.netlify.app/pdf/HSVmetricsCampaignsDarknessPOQFINAL.pdf&#34;&gt;corrected version of the original study&lt;/a&gt;;&lt;/li&gt;
&lt;li&gt;show how I found those problems with the original study in the course of conducting this replication;&lt;/li&gt;
&lt;li&gt;circulate these generally confirmatory findings, along with a pooled analysis revealing a stronger effect among conservatives; and&lt;/li&gt;
&lt;li&gt;provide a demonstration of how replication almost always enhances our knowledge about the original research, which I hope may encourage others to invest the time and money in such efforts.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;First some context.&lt;/p&gt;
&lt;p&gt;The original study that formed the basis of the manuscript shows that more negative campaign ads in 2008 were also more likely to contain darker images of President Obama. In 2009 when I started this work, I was most proud of the method to collect data on skin complexion outlined in study 1.  I included another study, what&#39;s now study 3, which shows that 2012 ANES survey-takers were more likely to respond negatively to Chinese characters after being presented with darker images of Obama (this is called the &lt;a href=&#34;http://onlinelibrary.wiley.com/doi/10.1111/spc3.12148/abstract&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Affect Misattribution Procedure (AMP)&lt;/a&gt;).
But the AMP was not a true experiment and a reviewer was concerned that Study 3 did not provide sufficiently rigorous, causal evidence that darker images alone can cause negative affect.  So I conducted an experiment that would establish a causal link between darker images of Obama and something I thought was even more important---stereotype activation. There were strong reasons to expect this effect based on past lab studies showing links between &lt;a href=&#34;https://ase.tufts.edu/psychology/tuscLab/documents/pubsCognitive2002.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;darker skin&lt;/a&gt; and &lt;a href=&#34;https://www.ncbi.nlm.nih.gov/pubmed/12088132&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;negative stereotypes about Blacks&lt;/a&gt;, and past observational studies showing far more &lt;a href=&#34;https://wcfia.harvard.edu/files/wcfia/files/2007_27_hochschild.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;negative socioeconomic outcomes across the board among darker versus lighter skinned Black Americans&lt;/a&gt;. We found an effect and published the three studies.&lt;/p&gt;
&lt;p&gt;This replication effort was prompted by a post-publication reanalysis and &lt;a href=&#34;http://www.ljzigerell.com/?p=3622&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;critique&lt;/a&gt;, which raised questions about potential weaknesses in the original analysis. My aim in replicating the study was to bring new data to the discussion and make sure we hadn&amp;rsquo;t polluted the literature with a false discovery.&lt;/p&gt;
&lt;p&gt;The main objection was the way we formed our stereotype consistency index. The items assessing stereotype consistency comprised 11 words with missing blank spaces (e.g., L A _ _). Each fragment had as one possible solution a stereotype-related completion. The complete list follows: L A _ _ (LAZY): C R _ _ _ (CRIME); _ _ O R (POOR); R _ _ (RAP); WEL _ _ _ _ (WELFARE); _ _ C E (RACE); D _ _ _ Y (DIRTY); B R _ _ _ _ _ (BROTHER); _ _ A C K (BLACK); M I _ _ _ _ _ _ (MINORITY); D R _ _ (DRUG).&lt;/p&gt;
&lt;p&gt;The author pointed out that there were many potential ways to analyze the original data---he claimed over 16 thousand. Yet very few of these are consistent with generally accepted research practices. We&#39;ve known, arguably since the &lt;a href=&#34;http://www.jstor.org/stable/2333051?origin=crossref&amp;amp;seq=1#page_scan_tab_contents&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;16th century&lt;/a&gt;, that combining several measures reduces measurement error and hence variance in estimation. This is particularly important in social science, and especially for this particular study---it would be unwise to attempt to use a single word completion or an arbitrary subset thereof to measure a complex, noisy construct like stereotype activation as measured via a word completion game. Rather, taking the &lt;a href=&#34;https://web.stanford.edu/~jrodden/issues_apsr.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;average or constructing an index based on clustering several measures&lt;/a&gt; should be expected to result in far less measurement error, which is what we did.&lt;/p&gt;
&lt;p&gt;Still, I am sympathetic to concerns about the &lt;a href=&#34;http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;garden of forking paths&lt;/a&gt;, which is part of the motivation for this replication.&lt;/p&gt;
&lt;p&gt;In the original study, I formed this index based on what I judged to be the most unambiguously negative word-completions (lazy, dirty, poor), consistent with past work suggesting that darker complexion activates the &lt;a href=&#34;https://ase.tufts.edu/psychology/tuscLab/documents/pubsCognitive2002.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;most &lt;/a&gt;&lt;a href=&#34;https://www.ncbi.nlm.nih.gov/pubmed/12088132&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;negative&lt;/a&gt; stereotypes about Blacks. I calculated that these were the three variables that also maximized interclass correlation (ICC). As a robustness check, I also computed a measure that maximized alpha reliability (AR). This measure contained more items, and also seemed to include stereotype-consistent word completions that were on balance negative---lazy, dirty, poor, crime, black, and welfare. I should have but did not report results based on a simple average of these items, which was not conclusive.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;http://www.ljzigerell.com/?p=3622&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;The critical reanalysis&lt;/a&gt; cited above shows a handful of statistically significant patterns that are inconsistent with the expectations in the original study, which is suggestive evidence that it&#39;s quite possible to find signal in noise if you&#39;re analyzing arbitrary sets of variables with the originally collected data. However, as shown below in the much larger replication sample below, none of these patterns replicate.&lt;/p&gt;
&lt;p&gt;The critique also noted that we did not include an analysis of several trailing questions we included on the original survey. The concern is the &lt;a href=&#34;http://science.sciencemag.org/content/345/6203/1502&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;file drawer problem&lt;/a&gt; - the incentives against and frequent failure to report null results - which obscures knowledge and is bad for the scientific enterprise.&lt;/p&gt;
&lt;p&gt;I included those measures based on &lt;a href=&#34;https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1643225&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;past work&lt;/a&gt; using the same images as stimuli, which found that darker images prompted more negative evaluations of Obama among people with more negative associations with Blacks, as measured using the Implicit Associations Test (IAT). But testing a specification that conditioned on our main outcome of interest---stereotype-consistent word completions---would mean conditioning on a post-treatment variable, particularly worrisome since we saw an effect on stereotype activation in the study.&lt;/p&gt;
&lt;p&gt;Below, I pool the data and report another specification that does not require us to condition on post-treatment variables. It takes advantage of the fact that conservatives had significantly higher levels of stereotype activation (which was documented in the original study), and shows that the effect is in fact stronger among this subgroup, providing preliminary evidence in favor of this hypothesis.&lt;/p&gt;
&lt;p&gt;The remainder of this post will present my own reanalysis of the original data, the replication, and finally some additional analysis of the data now possible with the larger, pooled data set.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Re-analysis of original data&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In the process of collecting data for the replication studies, I used the same interface, simply appending the new data as additional respondents completed the survey experiment. When I geo-coded the IP address data in the full data set, I found a discrepancy between the cases I originally geo-coded as U.S. cases, and the cases that now resolved to U.S. locations in the complete data set.  Many of these respondents appeared in sequence, suggesting they may have been skipped, perhaps due to issues related to connectivity to the geo-location server I used.&lt;/p&gt;
&lt;p&gt;This prompted me to conduct a full re-analysis of the data, which yields smaller estimates of stereotype activation. First, re-estimating the index yielded different items---&#39;black&#39; in place of &#39;dirty&#39; for the ICC measure and &#39;race&#39; in place of &#39;welfare&#39;, &#39;crime&#39;, and &#39;dirty&#39; in the AR measure. This is due in part to the way I computed the original indices and in part due to correcting the geo-coding issue. In the original study, I computed the index of variables that maximized alpha and ICC by hand because the epiCalc::alphaBest function (now epiDisplay::alphaBest) does not return results (nor an error message) for these data. For reanalysis, I wrote a function that computed variables to include in the index via successive removal of items. The overall alpha is actually slightly lower in new AR measure, while the new ICC measure has a slightly higher correlation coefficient.&lt;/p&gt;
&lt;p&gt;For the sake of transparency, I first report results based on the original items included in the index as reported in Messing et al. 2015 using the updated data, then report the new ICC and AR measures.&lt;/p&gt;
&lt;p&gt;Using the original indices with the errantly remove cases included, instead of a 36% increase in stereotype-consistent word completions using the ICC measure, this meant a revised estimate of a 20% increase in stereotype activation (M_Light = 0.33, M_Dark = 0.41, T(859.0) = 2.08, P = 0.038, two-sided). For the AR measure, instead of a 13% increase (M_Light = 0.97, M_Dark = 1.11, T(626.72) = 1.77, P = 0.078, two-sided), this meant an 8% increase (M_Light = 0.98, M_Dark = 1.06, T(850.9) = 1.12, P = 0.265, two-sided).&lt;/p&gt;
&lt;p&gt;Re-estimating the indices when including all U.S. cases translates to less conclusive findings---a revised estimate of an 8% increase in stereotype activation in the original study (M_Light = 0.79, M_Dark = 0.86, T(850.7) = 1.27, P = 0.203, two-sided) using the ICC measure, and an 8% increase (M_Light = 0.87, M_Dark = 0.91, T(839.1) = 0.77, P = 0.439, two-sided) using the AR measure.&lt;/p&gt;
&lt;p&gt;A slightly smaller effect was also observed when examining differences between conservatives and other participants. Correcting the geo-coding error and updating the indices reduced the estimate of stereotype activation for conservatives. Instead of a 53% increase, the original ICC measure yields a 29% increase (M_Other = 0.35, M_Conservative = 0.49, T(205.9) = 2.49, P = 0.013, two-sided).  The new ICC measure yields an 18% increase (M_Other = 0.80, M_Conservative = 0.98, T(207.9) = 2.41, P = 0.017, two-sided).  For the AR measure, instead of a 29% increase, this meant an 18% increase using either measure (original: M_Other = 0.99, M_Conservative = 1.18, T(210.4) = 2.11, P = 0.036, two-sided) (new: M_Other = 0.86, M_Conservative = 1.05, T(214.3) = 2.47, P = 0.014, two-sided).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The replication&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;I conducted one exact replication and one very close replication with slightly different images, which I pooled for a total of 3,151 respondents, substantially more than the 630 included in the original writeup.  This gives me more statistical power and more precise estimates of the effect in question. (I provide results for each design separately - one of which appears underpowered - at the end of this post).&lt;/p&gt;
&lt;p&gt;To be clear, I did not pre-register this replication. However, I&#39;ve tried to err on the side of exhaustive reporting when the original study did not provide exacting specificity in analyzing the new data. Due to the nature of this replication---the presentation of the same analysis conducted in the original study---the p-values provide highly informative, if not conclusive evidence regarding the nature of the effect.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/cb54a931e7c01f3bfd03caa2899518b447ab470e.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;The average reported age was 36; 52% of participants identified as female; 84% identified as White, 8% as Black; 5% as Hispanic; and 3% as Other. 52% identified as liberal, 27% as moderate, 22% as conservative.&lt;/p&gt;
&lt;p&gt;Recomputing the ICC index yielded the following items: black, poor, drug. Recomputing the AR index yielded: lazy, black, poor, welfare, crime, drug, which is close to the original study.&lt;/p&gt;
&lt;p&gt;In the replication data, the ICC yields a 5% increase in stereotype activation (M_Light = 0.90, M_Dark = 0.95, T(3142.8) = 1.56, P = 0.119, two-sided). Similarly, the alpha measure yields a 5% increase (M_Light = 1.04, M_Dark = 1.09, T(3145.8) = 1.70, P = 0.089, two-sided).&lt;/p&gt;
&lt;p&gt;The original study isn&#39;t completely clear on the question of whether a replication should report on the recomputed ICC and AP measures, or the exact same items as in the original study, so it&#39;s worth reporting those as well. The &lt;em&gt;original&lt;/em&gt; ICC measure yields a 3% increase in stereotype activation (M_Light = 0.36, M_Dark = 0.37, T(3148.9) = 0.51, P = 0.611, two-sided). Using the &lt;em&gt;original&lt;/em&gt; AR measure yields a 6% increase (M_Light = 1.01, M_Dark = 1.07, T(3147.9) = 1.91, P = 0.057, two-sided).&lt;/p&gt;
&lt;p&gt;Finally, it&#39;s worth reporting on an index that simply uses all stereotype-consistent items in the replication reveals a 5% increase in stereotype activation (M_Light = 1.30, M_Dark = 1.37, T(3147.7) = 2.05, P = 0.040, two-sided).&lt;/p&gt;
&lt;p&gt;A pooled analysis, after normalizing the ICC and AR measures, yields similar results:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/a0415aa41faee2db8452e14712ae9505f7a96f0a.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;=================================================
                      ICC     Alpha      ALL
-------------------------------------------------
  (Intercept)       -0.041   -0.028    1.292***
                    (0.034)  (0.034)  (0.036)
  cond: Dark/Light   0.067*   0.058    0.067*
                    (0.032)  (0.032)  (0.034)
  study              0.006   -0.001    0.008
                    (0.023)  (0.023)  (0.025)
-------------------------------------------------
  R-squared             0.0      0.0       0.0
  N                  4012     4012      4012
=================================================
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I also replicated this study with a different, lesser-known Black politician (Jesse White). However, a manipulation check revealed that only 36% of respondents said the candidate was Black in the &amp;ldquo;light&amp;rdquo; condition, compared to 83% in the darker condition, suggesting that any analysis would be severely confounded by perceived race of the target politician. (I did not ask this question in the Barack Obama studies).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Additional analysis&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The superior power afforded by pooling all three studies may allow the exploration of treatment heterogeneity. &lt;a href=&#34;https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1643225&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Past work&lt;/a&gt; suggests the possibility that darker images might cause people inclined toward more stereotype-consistent responses to evaluate politicians more negatively. However, this analysis would &lt;a href=&#34;http://andrewgelman.com/2017/09/12/conditioning-post-treatment-variables-can-ruin-experiment/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;condition on post-treatment variables&lt;/a&gt;, which in this case is particularly concerning since the treatment affects stereotype activation according to the original study and replication above.  As an alternative, I consider a specification that uses conservative identification instead, which is a strong predictor of stereotype activation (as shown in the original study), but shouldn&amp;rsquo;t be affected by the treatment. It reveals evidence for the predicted interactions, suggesting that when conservatives are exposed to darker rather than lighter images of Obama, they have slightly &amp;ldquo;colder&amp;rdquo; feelings toward the former president (P = 0.039), perceive him to be less competent (P = 0.061), and less trustworthy (P = 0.083).&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;=================================================================
                             obama_therm  competence    trust
-----------------------------------------------------------------
  (Intercept)                 69.446***    4.213***    3.867***
                              (0.705)     (0.029)     (0.031)
  cond: Dark/Light             0.451       0.024       0.045
                              (0.989)     (0.041)     (0.044)
  iscons                     -40.497***   -1.574***   -1.625***
                              (1.565)     (0.064)     (0.069)
  cond: Dark/Light x iscons   -4.462*     -0.167      -0.165
                              (2.160)     (0.089)     (0.095)
-----------------------------------------------------------------
  R-squared                        0.3         0.3         0.2
  N                             3932        3928        3926
=================================================================
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A plot of the model predictions for the thermometer ratings suggests that the effect is concentrated among conservatives.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/f82207e502f7cdd9c8a6b178efa7ca801d1b8e4c.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The more items one uses to form an index, the less noise we should expect, and the more likely any replication attempt should be expected to succeed.  It should also mean greater statistical precision. This could explain the remaining discrepancy between this study and the original after adjusting for the geo-coding error pointed out above. It&#39;s also possible that something about the timing or the subjects recruited in the replication studies that explain the observed differences.&lt;/p&gt;
&lt;p&gt;Nonetheless, this replication provides evidence that darker images of Black political figures, or at least of President Barack Obama, do in fact activate stereotypes.  This much larger sample suggests that the true effect is smaller than what I found in the original study, which as noted above, contained some errors.&lt;/p&gt;
&lt;p&gt;Replication materials available on &lt;a href=&#34;http://dx.doi.org/10.7910/DVN/WY7PR8&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;dataverse&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Appendix&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Below I present alternate specifications estimated without pooling. These specifications suggest first that replication 2 (as well as the original study) was not well-powered. It also suggests that the outcome measures with more items yield more reliable estimates.&lt;/p&gt;
&lt;p&gt;Outcome measure summing all items:&lt;/p&gt;
&lt;p&gt;Replication 1: M_Light = 1.29, M_Dark = 1.36, T(2115.4) = -1.59, P = 0.113, two-sided&lt;/p&gt;
&lt;p&gt;Replication 2: M_Light = 1.30, M_Dark = 1.39, T(982.7) = -1.35, P = 0.177, two-sided&lt;/p&gt;
&lt;p&gt;Original Alpha outcome measure:&lt;/p&gt;
&lt;p&gt;Replication 1: M_Light = 0.99, M_Dark = 1.06, T(2114.9) = -1.79, P = 0.073, two-sided&lt;/p&gt;
&lt;p&gt;Replication 2: M_Light = 1.04, M_Dark = 1.09, T(979.8) = -0.85, P = 0.393, two-sided&lt;/p&gt;
&lt;p&gt;Newly estimated Alpha outcome measure:&lt;/p&gt;
&lt;p&gt;Replication 1: M_Light = 1.02, M_Dark = 1.09, T(2109.4) = -1.77, P = 0.077, two-sided&lt;/p&gt;
&lt;p&gt;Replication 2: M_Light = 1.07, M_Dark = 1.10, T(984.6) = -0.53, P = 0.597, two-sided&lt;/p&gt;
&lt;p&gt;Original ICC outcome measure:&lt;/p&gt;
&lt;p&gt;Replication 1: M_Light = 0.89, M_Dark = 0.94, T(2100.6) = -1.49, P = 0.137, two-sided.&lt;/p&gt;
&lt;p&gt;Replication 2: M_Light = 0.93, M_Dark = 0.96, T(985.9) = -0.67, P = 0.506, two-sided&lt;/p&gt;
&lt;p&gt;Newly estimated ICC outcome measure&lt;/p&gt;
&lt;p&gt;Replication 1: M_Light = 0.35, M_Dark = 0.36, T(2122.5) = -0.22, P = 0.826, two-sided&lt;/p&gt;
&lt;p&gt;Replication 2: M_Light = 0.38, M_Dark = 0.40, T(978.7) = -0.72, P = 0.472, two-sided
 
A replication of prior &lt;a href=&#34;http://www.ljzigerell.com/?p=3622&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;critique and reanalysis&lt;/a&gt;.  The patterns that run contrary to our original findings are not significant in the replication data.&lt;/p&gt;
&lt;table style=&#34;height:1612px;&#34; width=&#34;584&#34;&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Variable&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;effect size&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;p-value&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;feeling therm&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;-0.028&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.439&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;race minority welfare crime rap&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.022&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.538&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;race minority welfare rap&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.017&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.629&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;race minority rap&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.011&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.752&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;race&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.006&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.861&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;minority&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;-0.013&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.713&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;rap&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.024&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.502&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;welfare&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.014&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.695&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;comp&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;-0.013&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.72&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;crime&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.016&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.659&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;trust&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.002&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.946&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;brother&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.042&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.236&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;drug&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.008&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.822&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;lazy&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.026&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.474&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;black&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.084&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.019&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;dirty&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.028&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.438&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;poor&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;-0.002&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.957&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;allwcs&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.073&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.04&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;original&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.018&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.611&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;alpha&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.068&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;&lt;span style=&#34;font-weight:400;&#34;&gt;0.057&lt;/span&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;</description>
    </item>
    
    <item>
      <title>Estimating heterogeneous treatment effects and the effects of heterogeneous treatments with ensemble methods</title>
      <link>https://solmessing.netlify.app/publication/grimmer-2017-estimating/</link>
      <pubDate>Sun, 01 Jan 2017 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/grimmer-2017-estimating/</guid>
      <description>&lt;pre&gt;&lt;code&gt;- [Replication materials](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/BQMLQW).
- [Software implementation](https://github.com/SolomonMg/HetSL) (under development)
&lt;/code&gt;&lt;/pre&gt;
</description>
    </item>
    
    <item>
      <title>Partisan Conflict and Congressional Outreach</title>
      <link>https://solmessing.netlify.app/publication/solomon-2017-partisan/</link>
      <pubDate>Sun, 01 Jan 2017 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/solomon-2017-partisan/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Bias in the flesh: Skin complexion and stereotype consistency in political campaigns</title>
      <link>https://solmessing.netlify.app/publication/messing-2016-bias/</link>
      <pubDate>Fri, 01 Jan 2016 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/messing-2016-bias/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Ideologically diverse news, an agenda for future research</title>
      <link>https://solmessing.netlify.app/post/exposure-to-ideologically-diverse-response/</link>
      <pubDate>Fri, 24 Apr 2015 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/exposure-to-ideologically-diverse-response/</guid>
      <description>&lt;p&gt;Earlier this month, we published an early access version of our paper in ScienceExpress (&lt;a href=&#34;https://solmessing.netlify.app/pdf/Science-2015-Bakshy-1130-2.pdf&#34;&gt;Bakshy et al. 2015&lt;/a&gt;), &amp;ldquo;Exposure to ideologically diverse news and opinion on Facebook.&amp;rdquo; The paper constitutes the first attempt to quantify the extent to which ideologically cross-cutting hard news and opinion is shared by friends, appears in algorithmically ranked News Feeds, and is actually consumed (i.e., click through to read).&lt;/p&gt;
&lt;p&gt;We are grateful for the widespread interest this paper, which grew out of two threads of related research that we began nearly five years ago: Eytan and Lada&#39;s work on the role of social networks in information diffusion (&lt;a href=&#34;http://arxiv.org/pdf/1201.4145v2.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bakshy et al. 2012&lt;/a&gt;) and Sean and Solomon&#39;s work on selective exposure in social media (&lt;a href=&#34;http://crx.sagepub.com/content/41/8/1042&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Messing and Westwood 2012&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;While &lt;em&gt;Science&lt;/em&gt; papers are explicitly prohibited from suggesting future directions for research, we would like to shed additional light on our study and raise a few questions that we would be excited to see addressed in future work.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Tradeoffs when Selecting a Population&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;There were tradeoffs when deciding on who to include in this study. While we could have examined all U.S. adults on Facebook, we focused on people who identify as liberals or conservatives and encounter hard news, opinion, and other political content in social media regularly. We did so because many important questions around &amp;ldquo;echo chambers&amp;rdquo; and &amp;ldquo;filter bubbles&amp;quot;on Facebook relate to this subpopulation, and we used self-reported ideological preferences to define it.&lt;/p&gt;
&lt;p&gt;Using self-reported ideological preferences in online profiles is not the only a way to measure ideology or define the population of interest. Yet, people who publicly identify as liberals or conservatives in their Facebook profiles are an interesting and important subpopulation worthy of study for many reasons. As &lt;a href=&#34;http://gking.harvard.edu/files/gking/files/words.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hopkins and King 2010&lt;/a&gt; have pointed out, studying the expression and behavior of those who are politically engaged online is of interest to political scientists studying activists (&lt;a href=&#34;http://www.hup.harvard.edu/catalog.php?isbn=9780674942936&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Verba, Schlozman, and Brady 1995&lt;/a&gt;), the media (&lt;a href=&#34;http://195.130.87.21:8080/dspace/handle/123456789/979&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Drezner and Farrell 2004&lt;/a&gt;), public opinion (&lt;a href=&#34;https://scholar.google.com/scholar?cluster=15572370959137190849&amp;amp;hl=en&amp;amp;as_sdt=2005&amp;amp;sciodt=0,5&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Gamson 1992&lt;/a&gt;), social networks (&lt;a href=&#34;http://dl.acm.org/citation.cfm?id=1134277&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Adamic and Glance 2005&lt;/a&gt;; &lt;a href=&#34;http://www.langtoninfo.co.uk/web_content/9780521542234_frontmatter.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Huckfeldt and Sprague 1995&lt;/a&gt;), and elite influence (&lt;a href=&#34;http://press.princeton.edu/titles/8425.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Grindle 2005&lt;/a&gt;; &lt;a href=&#34;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.160.8347&amp;amp;rep=rep1&amp;amp;type=pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hindman, Tsioutsiouliklis, and Johnson 2003;&lt;/a&gt; &lt;a href=&#34;https://books.google.com/books?hl=en&amp;amp;lr=&amp;amp;id=83yNzu6toisC&amp;amp;oi=fnd&amp;amp;pg=PR8&amp;amp;dq=The&amp;#43;Nature&amp;#43;and&amp;#43;Origins&amp;#43;of&amp;#43;Mass&amp;#43;Opinion&amp;amp;ots=6oEwiBZtSM&amp;amp;sig=ySnwRZUuVtOyTkC-3raaL88pYns#v=onepage&amp;amp;q=The%20Nature%20and%20Origins%20of%20Mass%20Opinion&amp;amp;f=false&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Zaller 1992&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;This subpopulation has limitations and is not the only population of interest. The data are not appropriate for those who seek estimates of the entire U.S. public, people without strong opinions, or people not on Facebook (at least not without additional extrapolation, re-weighting, additional evidence, etc.). While our data &lt;em&gt;could plausibly&lt;/em&gt; also provide good estimates of the population of people who are ideologically active and have clear preferences, we are not claiming that&#39;s necessarily the case---that remains to be determined in future work.&lt;/p&gt;
&lt;p&gt;We&#39;d like to help other researchers looking to study other populations understand more about the population we&#39;ve defined. An important question in this regard is what proportion of active U.S. adults actually report an identifiable left/right/center ideology in their profile. That number is 25%, or 10.1 million people.&lt;/p&gt;
&lt;p&gt;It&#39;s also informative to examine the proportion of those users who provide identifiable profile affiliations conditional on demographics and Facebook usage:&lt;/p&gt;
&lt;table style=&#34;border-collapse:collapse;width:473px;height:512px;&#34; border=&#34;0&#34; width=&#34;174&#34; cellspacing=&#34;0&#34; cellpadding=&#34;0&#34;&gt;&lt;colgroup&gt; &lt;col style=&#34;width:65pt;&#34; span=&#34;2&#34; width=&#34;87&#34; /&gt; &lt;/colgroup&gt;
&lt;tbody&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;width:65pt;&#34; width=&#34;87&#34; height=&#34;24&#34;&gt;&lt;strong&gt;Age&lt;/strong&gt;&lt;/td&gt;
&lt;td class=&#34;xl63&#34; style=&#34;width:65pt;&#34; width=&#34;87&#34;&gt;&lt;strong&gt;Percent reporting ideological affiliation&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;18-24&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;21.60%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;25-44&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;28.50%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;45-64&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;24.30%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;65+&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;21.40%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;&lt;/td&gt;
&lt;td class=&#34;xl63&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;&lt;strong&gt;Gender&lt;/strong&gt;&lt;/td&gt;
&lt;td class=&#34;xl63&#34;&gt;&lt;strong&gt;Percent reporting ideological affiliation&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;Female&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;21.90%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;Male&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;30.60%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;&lt;/td&gt;
&lt;td class=&#34;xl63&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;&lt;strong&gt;Login Days&lt;/strong&gt;&lt;/td&gt;
&lt;td class=&#34;xl63&#34;&gt;&lt;strong&gt;Percent reporting ideological affiliation&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;105-140&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;18.90%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr style=&#34;height:18pt;&#34;&gt;
&lt;td class=&#34;xl63&#34; style=&#34;height:18pt;&#34; height=&#34;24&#34;&gt;140-185&lt;/td&gt;
&lt;td class=&#34;xl64&#34;&gt;26.70%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Clearly those who report an ideology in their profile tend to be more active on Facebook. They are also more likely to be men, which is consistent with the well-documented gender gap in American politics (&lt;a href=&#34;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=245226&amp;amp;fileId=S0003055404001315&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Box-Steffensmeier 2004&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;It&#39;s possible that these individuals differ from other Facebook users in other ways. It seems plausible to expect these people to have higher levels of political interest, a stronger sense of political ideology and political identity, and to be more likely to be active in politics than most others on Facebook. It&#39;s also possible that these individuals are more extroverted than the average user, especially in the somewhat taboo domain of politics. These possibilities also strike us as interesting questions for study in future work.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;How to Measure Ideology&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;We hope others will replicate this work using other populations and ways of measuring ideology, which will provide a broader view of exposure to political media. Data on ideology could be collected by, for example, surveying users, imputing ideology based on user behavior, or joining data to the voter file. Each of these methods have advantages and potential challenges.&lt;/p&gt;
&lt;p&gt;Using surveys in future work would allow researchers to collect data on ideology in a way that can facilitate comparisons with much of the extant literature in political science, and allow researchers to sample from a less politically engaged population. Of course, this could be tricky because survey response rates might be affected by the phenomenon under study. In other words, the salience of political discussion from the right or left, and/or prior choices to consume content could make people more/less likely to respond to a survey asking about ideology, or affect the way they report the strength of their ideological preferences. This could confound measurement in a way that would be difficult to detect and correct. Yet it would be fascinating to see how survey results compare to the results in this study.&lt;/p&gt;
&lt;p&gt;We would also encourage the application of large-scale methods that impute individuals&amp;rsquo; ideological leanings using social networks or revealed preferences. This would have the advantage of allowing researchers to estimate ideological preferences for a broader population, and could be applied to empirical contexts for which self-reported ideological affiliations are not present.&lt;/p&gt;
&lt;p&gt;However, these approaches present challenges. Imputing ideology based on social networks would make it difficult to estimate what proportion of people&amp;rsquo;s networks contain individuals from the other side. &lt;a href=&#34;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=9586211&amp;amp;fileId=S0003055414000525&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bond and Messing, 2015&lt;/a&gt; and &lt;a href=&#34;http://pan.oxfordjournals.org/content/23/1/76.full.pdf?keytype=ref&amp;amp;ijkey=uMFPw4dsMHM7608&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Barberá 2014&lt;/a&gt; discuss some of the challenges related to estimating ideology based on revealed preferences. Another challenge specific to the quantities estimated in our paper is that because behavior may be caused by the composition of individuals&amp;rsquo; social networks, what their friends share, and how they engage with Facebook, using revealed preferences to select the population could introduce endogenous selection bias (&lt;a href=&#34;http://www.annualreviews.org/doi/abs/10.1146/annurev-soc-071913-043455&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Elwert and Winship 2014&lt;/a&gt;). A study that negotiates these issues would be a tremendously valuable contribution. Similar methods could also be used to obtain measures of ideological alignment of content.&lt;/p&gt;
&lt;p&gt;Lastly, researchers could use party registration from the voter file. This approach would yield millions of records, but have different selection problems&amp;mdash;match rates may differ by region, state, age, gender, etc. Again, the advantage of approaches like this are that these studies compliment each other and provide a fuller picture of how exposure to viewpoints from the other side occur in social media.&lt;/p&gt;
&lt;p&gt;Future work should also examine how exposure varies in different subpopulations. For example, one hypothesis to test is whether those with weaker or less consistent ideological preferences have more cross cutting content shared by friends, rendered in social media streams, and selected for reading. Some preliminary analysis suggests that indeed, among the individuals in our study, those with a weaker stated ideological affiliation have on average more cross-cutting content at each stage in the exposure process.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/5a469dbbfb5b2916a87554ca4a247579da11e2b2.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Other Data Sources&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;There are many other important questions related to this paper that necessitate new data sources: Does encountering cross-cutting content increase or decrease attitude polarization? What about attitudes toward members of the other side? Does it change specific policy preferences? Are liberals and conservatives more or less likely to see content in News Feed &lt;em&gt;because&lt;/em&gt; it was cross-cutting? Do they actively avoid cross-cutting political content &lt;em&gt;because&lt;/em&gt; of expressions in the title or because of the fact that the media source is suggestive of a cross-cutting article? How do changes to ranking algorithms and user interfaces affect selective exposure? And how can we better understand actual discourse about politics in social media, rather than merely shared media content?&lt;/p&gt;
&lt;p&gt;Answering these questions necessitates collecting innovative data sets via online experimentation (&lt;a href=&#34;http://pan.oxfordjournals.org/content/20/3/351.short&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Berinsky et al. 2012&lt;/a&gt;), social media (&lt;a href=&#34;http://scholar.harvard.edu/dtingley/files/fall2012.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Ryan and Broockman 2012&lt;/a&gt;), crowdsourcing (&lt;a href=&#34;http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2526461&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Budak et al. 2014&lt;/a&gt;), large scale field experimentation (&lt;a href=&#34;http://gking.harvard.edu/publications/randomized-Experimental-Study-Censorship-China&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;King et al. 2014&lt;/a&gt;), observational social media data, clever ways to collect data about individual differences in ranking (&lt;a href=&#34;http://www.lazerlab.net/sites/default/files/publications/Measuring%20Personalization%20of%20Web%20Search.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hannak et al. 2013&lt;/a&gt;), smart ways to combine behavioral and survey data (&lt;a href=&#34;http://arxiv.org/pdf/1304.1837v3.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Chen et al. 2014&lt;/a&gt;), and panel data (&lt;a href=&#34;http://faculty-gsb.stanford.edu/athey/documents/localnews.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Athey and Mobius 2012&lt;/a&gt;, &lt;a href=&#34;https://5harad.com/papers/bubbles.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Flaxman et al. 2014&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;Many of these are causal questions necessitating experimental and/or quasi experimental designs. For example, the extent to which people select content because it is cross-cutting could be investigated using experiments like this one (e.g., &lt;a href=&#34;http://crx.sagepub.com/content/41/8/1042&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Messing and Westwood 2012&lt;/a&gt;) or through identifying sources of natural exogenous variation. And while Diana Mutz and others have done ground-breaking research on the effects of encountering cross-cutting arguments on political attitudes (&lt;a href=&#34;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=208463&amp;amp;fileId=S0003055402004264&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mutz 2002&lt;/a&gt;b) and behavior (&lt;a href=&#34;http://www.jstor.org/stable/3088437&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mutz 2002&lt;/a&gt;a), more research into how these effects play out in the long term (using approaches like &lt;a href=&#34;http://faculty.wcas.northwestern.edu/~jnd260/pub/Druckman%20Fein%20Leeper%20APSR.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Druckman et al 2012&lt;/a&gt;) would be of tremendous benefit to the literature. It is difficult to expose people to any sort of argument for a long period of time (say over the course of a U.S. national political campaign cycle), in a way that is not confounded with people&#39;s existing preferences and the social environment, though creative quasi-experimental work (&lt;a href=&#34;http://web.stanford.edu/~ayurukog/cable_news.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Martin and Yurukoglu 2014&lt;/a&gt;) is emerging in this area.&lt;/p&gt;
&lt;p&gt;Many of these questions necessitate that researchers identify the effects of cross-cutting arguments both on and off Facebook. To get a full picture of how cross-cutting arguments affect politics requires understanding the myriad of ways individuals get information, both on the Internet (&lt;a href=&#34;https://5harad.com/papers/bubbles.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Flaxman et al. 2014&lt;/a&gt;) and offline (&lt;a href=&#34;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=208463&amp;amp;fileId=S0003055402004264&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mutz 2002&lt;/a&gt;a), what kinds of information people discuss in offline contexts (&lt;a href=&#34;http://dx.doi.org/10.1017/S0003055402004264&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mutz 2002&lt;/a&gt;b), and the relative influence of all of these factors on opinions.&lt;/p&gt;
&lt;p&gt;Finally, if individuals&#39; online networks and choices do substantially impact the diversity of news in individuals&#39; overall &amp;ldquo;information diets,&amp;rdquo; future research could examine the effects of connecting those with more disparate views (&lt;a href=&#34;http://ajps.org/2015/03/11/partisanship-in-social-settings-when-democrats-and-republicans-meet/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Klar 2014&lt;/a&gt;), encouraging consumption of cross-cutting content (&lt;a href=&#34;http://www.smunson.com/portfolio/projects/socnews_icwsm15.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Agapie and Munson 2015&lt;/a&gt;), or simply encouraging individuals to read more diverse news by making individuals more aware of the balance of news they consume (&lt;a href=&#34;http://www.smunson.com/portfolio/projects/aggdiversity/balancer-icwsm.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Munson et al. 2013&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;These questions are especially important in light of the fact that there are substantial opportunities for people to read more news on Facebook. The plots below illustrate the average proportion of stories shared by friends, those that are seen in News Feed, and those clicked on for liberals and conservatives in the study. Clearly there is an opportunity to read more news from either side.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/ddb611f4a223213b3a653208e5bdd048d6b9cd94.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Dataverse&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Finally, we believe that reproducing, replicating, and conducting additional analyses on extant data sets is extremely important and helps generate ideas for future work (&lt;a href=&#34;http://gking.harvard.edu/files/abs/replication-Abs.shtml&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;King 1995&lt;/a&gt;, &lt;a href=&#34;http://thomasleeper.com/2015/05/open-science-language/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Leeper 2015&lt;/a&gt;). In that spirit, we have created a &lt;a href=&#34;https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/AAI7VA&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Dataverse archive&lt;/a&gt;. The repository includes replication data, scripts, as well as some additional supplementary data and code for extending our work.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;E. Bakshy, S. Messing, L.A. Adamic. 2015. Exposure to ideologically diverse news and opinion on Facebook. &lt;em&gt;Science&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;E. Bakshy, I. Rosenn, C.A. Marlow, L.A. Adamic. 2012. The Role of Social Networks in Information Diffusion. &lt;em&gt;ACM WWW 2012.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;S. Messing and S.J. Westwood. 2012. Selective Exposure in the Age of Social Media: Endorsements Trump Partisan Source Affiliation When Selecting News Online. &lt;em&gt;Communication Research&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;P. Barberá (2015). Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data. &lt;em&gt;Political Analysis&lt;/em&gt;, &lt;em&gt;23&lt;/em&gt;(1), 76-91.&lt;/p&gt;
&lt;p&gt;R. Bond, S. Messing, Quantifying Social Media&amp;rsquo;s Political Space: Estimating Ideology from Publicly Revealed Preferences on Facebook. &lt;em&gt;American Political Science Review&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;F. Elwert and C. Winship. 2014. Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable. &lt;em&gt;Annual Review of Sociology.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;G. King, J. Pan, and M. E. Roberts. 2014. Reverse-Engineering Censorship in China: Randomized Experimentation and Participant Observation. &lt;em&gt;Science&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;C. Budak, S. Goel, &amp;amp; J. M. Rao. (2014). Fair and Balanced? Quantifying Media Bias Through Crowdsourced Content Analysis. &lt;em&gt;Quantifying Media Bias Through Crowdsourced Content Analysis (November 17, 2014)&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;S. Athey, M. Mobius. The Impact of News Aggregators on Internet News Consumption: The Case of Localization. Working paper. &lt;a href=&#34;http://faculty-gsb.stanford.edu/athey/documents/localnews.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;http://faculty-gsb.stanford.edu/athey/documents/localnews.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;A. Hannak, P. Sapiezynski, A. Molavi Kakhki, B. Krishnamurthy, D. Lazer, A. Mislove, C. Wilson. 2013. Measuring personalization of web search. &lt;em&gt;ACM WWW 2013&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;A. Chen and A. Owen and M. Shi. Data Enriched Linear Regression. Working paper. &lt;a href=&#34;http://arxiv.org/pdf/1304.1837v3.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;http://arxiv.org/pdf/1304.1837v3.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;G.J. Martin, A. Yurukoglu. Working paper. Bias in Cable News: Real Effects and Polarization. Working paper. &lt;a href=&#34;http://web.stanford.edu/~ayurukog/cable_news.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;http://web.stanford.edu/~ayurukog/cable_news.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;S.R. Flaxman, S. Goel, J.M. Rao. Filter Bubbles, Echo Chambers, and Online News Consumption. Working paper. &lt;a href=&#34;https://5harad.com/papers/bubbles.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://5harad.com/papers/bubbles.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;D.C. Mutz. 2002. The Consequences of Cross-Cutting Networks for Political Participation. &lt;em&gt;American Journal of Political Science&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;D.C. Mutz. 2002. Cross-cutting Social Networks: Testing Democratic Theory in Practice. &lt;em&gt;American Political Science Review&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;J. N. Druckman, J. Fein, &amp;amp; T. Leeper. 2012. A source of bias in public opinion stability. &lt;em&gt;American Political Science Review&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;E. Agapie, S.A. Munson. 2015. &amp;ldquo;&lt;a href=&#34;http://smunson.com/portfolio/projects/socnews_icwsm15.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Social Cues and Interest in Reading Political News Stories&lt;/a&gt;.&amp;rdquo; &lt;em&gt;AAAI ICWSM 2015&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;S. Klar. 2014. Partisanship in a Social Setting. &lt;em&gt;American Journal of Political Science&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;S.A. Munson, S.Y. Lee, P. Resnick. 2013. Encouraging Reading of Diverse Political Viewpoints with a Browser Widget. &lt;em&gt;AAAI ICWSM 2013&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;G. King. 1995. &amp;ldquo;Replication, Replication.&amp;rdquo; &lt;em&gt;Political Science and Politics&lt;/em&gt;. &lt;a href=&#34;http://j.mp/1wP9Vqn&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;http://j.mp/1wP9Vqn&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;T. Leeper. 2015. What&#39;s in a Name? The Concepts and Language of Replication and Reproducibility. Blog post. &lt;a href=&#34;http://thomasleeper.com/2015/05/open-science-language/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;http://thomasleeper.com/2015/05/open-science-language/&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Exposure to ideologically diverse news and opinion on Facebook</title>
      <link>https://solmessing.netlify.app/publication/bakshy-2015-exposure/</link>
      <pubDate>Thu, 01 Jan 2015 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/bakshy-2015-exposure/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;http://science.sciencemag.org/content/348/6239/1090&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Review by David Lazer&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://science.sciencemag.org/highwire/filestream/630053/field_highwire_adjunct_files/1/Bakshy-SM.revision.1.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Supplementary materials&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://dx.doi.org/10.7910/DVN/LDJ7MS&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Replication materials&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Media: &lt;a href=&#34;http://www.nytimes.com/2015/05/08/technology/facebook-study-disputes-theory-of-political-polarization-among-users.html?_r=0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;New York Times&lt;/a&gt;,
the &lt;a href=&#34;http://www.washingtonpost.com/news/energy-environment/wp/2015/05/07/facebook-study-says-its-mainly-your-fault-not-theirs-that-you-read-things-you-already-agree-with/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Washington Post&lt;/a&gt;, the &lt;a href=&#34;http://www.bbc.com/news/science-environment-32606724&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;BBC&lt;/a&gt;, &lt;a href=&#34;http://www.cbsnews.com/news/facebooks-news-feed-limits-your-world-view/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CBS&lt;/a&gt;, &lt;a href=&#34;http://www.huffingtonpost.com/2015/05/12/facebook-study-polarization_n_7245192.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Huffington Post&lt;/a&gt;, &lt;a href=&#34;http://arstechnica.com/science/2015/05/dont-just-blame-facebook-we-build-our-own-bubbles/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Ars Technica&lt;/a&gt;, &lt;a href=&#34;http://www.wired.co.uk/news/archive/2015-05/08/facebook-echo-chamber-study&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Wired
UK&lt;/a&gt;,
&lt;a href=&#34;http://www.theverge.com/2015/5/7/8564795/facebook-online-opinion-filter-bubble-news-feed-study&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;The Verge&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Quantifying social media&#39;s political space: Estimating ideology from publicly revealed preferences on Facebook</title>
      <link>https://solmessing.netlify.app/publication/bond-2015-quantifying/</link>
      <pubDate>Thu, 01 Jan 2015 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/bond-2015-quantifying/</guid>
      <description></description>
    </item>
    
    <item>
      <title>When to Use Stacked Barcharts?</title>
      <link>https://solmessing.netlify.app/post/when-to-use-stacked-barcharts/</link>
      <pubDate>Sat, 11 Oct 2014 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/when-to-use-stacked-barcharts/</guid>
      <description>&lt;p&gt;Yesterday a few of us on Facebook&amp;rsquo;s Data Science Team released a &lt;a href=&#34;https://www.facebook.com/notes/10152581594083859/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;blogpost showing how candidates are campaigning on Facebook in the 2014 U.S. midterm elections&lt;/a&gt;. It was &lt;a href=&#34;http://www.washingtonpost.com/blogs/govbeat/wp/2014/10/10/how-candidates-use-facebook-motivation-more-than-persuasion/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;picked up in the Washington Post&lt;/a&gt;, in which &lt;a href=&#34;http://www.washingtonpost.com/people/reid-wilson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Reid Wilson&lt;/a&gt; calls us &amp;quot;data wizards.&amp;quot; Outstanding.&lt;/p&gt;
&lt;p&gt;I used &lt;a href=&#34;http://had.co.nz/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hadly Wickham&#39;s&lt;/a&gt; ggplot2 for every visualization in the post except a map that &lt;a href=&#34;http://web.stanford.edu/~arjunw/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Arjun Wilkins&lt;/a&gt; produced using D3, and for the first time I used stacked bar charts.  Now as I&#39;ve stated previously, &lt;a href=&#34;https://solmessing.netlify.app/post/visualization-series-insight-from-cleveland-and-tufte-on-plotting-numeric-data-by-groups&#34;&gt;one should generally avoid bar charts, and especially stacked bar charts&lt;/a&gt;, except in a few specific circumstances.&lt;/p&gt;
&lt;p&gt;But let&#39;s talk about when not to use stacked bar charts first---I had the pleasure of chatting with Kaiser Fung of &lt;a href=&#34;http://junkcharts.typepad.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JunkCharts&lt;/a&gt; fame the other day, and I think what makes his site so compelling is the mix of schadenfreude and &lt;a href=&#34;http://betterthanenglish.com/fremdscham-german/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Fremdscham&lt;/a&gt; that makes taking apart someone else&#39;s mistake such an effective teaching strategy and such a memorable read. I also appreciate the subtle nod to &lt;a href=&#34;https://en.wikipedia.org/wiki/Found_object&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;junk art&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Here&#39;s a typical, terrible stacked bar chart, which I found on &lt;a href=&#34;http://www.storytellingwithdata.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;http://www.storytellingwithdata.com/&lt;/a&gt; and originally published on a &lt;a href=&#34;http://blogs.wsj.com/digits/2012/10/22/microsoft-windows-8-forrester/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Wall Street Journal blogpost&lt;/a&gt;. It shows the share of the personal computing device market by operating system, over time. The problem with using a stacked bar chart is that there are only two common baselines for comparison (the top and bottom of the plotting area), but we are interested in the relative share for more than two OS brands. The post is really concerned with Microsoft, so one solution would be to plot Microsoft versus the rest, or perhaps Microsoft on top versus Apple on the bottom with &amp;quot;Other&amp;quot; in the middle. Then we&#39;d be able to compare the over time market share for Apple and Microsoft. As the author points out, an over time trend can also be visualized with line plots.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/b5e194c5114b79478a7ffcf600b18cd205a3a1b7.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;By far the worst offender I found in my 5 minute Google search was &lt;a href=&#34;http://junkcharts.typepad.com/junk_charts/2014/08/one-guaranteed-to-make-stephen-few-cry-.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;from junkcharts&lt;/a&gt; and originally published on &lt;a href=&#34;http://www.vox.com/2014/7/28/5944065/electric-cars-plug-in-vehicles-rising-sales-US&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Vox&lt;/a&gt;. These cumulative sum plots are so bad I was surprised to see them still up. The first problem is that the plots represent an attempt to convey way too much information---either plot total sales or pick a few key brands that are most interesting and plot them on a multi-line chart or set of faceted time series plots. The only brand for which you can quickly get a sense of sales over time is the Chevy Volt because it&#39;s on the baseline. I&#39;m sure the authors wanted to also convey the proportion of sales each year, but if you want to do that just plot the relative sales. Of course, the order in which the bars appear on the plot has no organizing principle, and you need to constantly move your eyes back and forth from the legend to the plot when trying to make sense of this monstrosity.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/a1281b5624937d4bf8069706e74028aeb8f9952d.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;As Kaiser notes in his post, less is often more. Here&#39;s his redux, which uses lines and aggregates by both quarter and brand, resulting in a far superior visualization:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/df59a98319dbf363b90514243d0ded9d9aa191ce.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;So when *should* you use a stacked bar chart? Here are a two scenarios with examples, inspired by work with &lt;a href=&#34;http://eytan.github.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Eytan Bakshy&lt;/a&gt; and conversations with &lt;a href=&#34;http://ta.virot.me/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Ta Chiraphadhanakul&lt;/a&gt; and &lt;a href=&#34;http://www.johnmyleswhite.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;John Myles White&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;1. You care about comparing the proportion of two things, in this case the share of posts by Democrats and Republicans, along a variety of dimensions.  In this case those dimensions consist of keyword (dictionary-based) categories (above) and LDA topics (below).  When these are sorted by relative proportion, the reader gains insight into which campaign strategies and issues are used more by Republican or Democratic candidates.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/b16c48bc6a47363ced0e685cb506df99e15e0a73.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;ol start=&#34;2&#34;&gt;
&lt;li&gt;You care about comparing proportions along an ordinal, additive variable such as 5-point party identification, along a set of dimensions.  I provide an example from a forthcoming paper below (I&#39;ll re-insert the axis labels once it&#39;s published).  Notice that it draws the reader toward two sets of comparisons across dimensions -- one for strong democrats and republicans, the other for the set of *all* Democrats and *all* Republicans.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/312fb516b818733414acbe395ded53b8424f37a2.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Of course, R code to produce these plots follows:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Uncomment these lines and install if necessary:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;ggplot2&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;dplyr&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#39;scales&amp;#39;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ggplot2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;dplyr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;scales&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# We start with the raw number of posts for each party for&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# each candidate. Then we compute the total by party and&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# category.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;group_by&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;party&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;all_cats&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summarise&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tot&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;summ&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;posts&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Next, compute the proportion by party for each category&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# using dplyr::mutate&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;group_by&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;all_cats&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;mutate&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;prop&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;tot&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;sum&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Now compute the difference by category and order the&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# categories by that difference:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;group_by&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;all_cats&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;mutate&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;pdiff&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;diff&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;prop&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;all_cats&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;reorder&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;all_cats&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;pdiff&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# And plot:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;catsByParty&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;all_cats&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;y&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;prop&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;fill&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;party&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_y_continuous&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;labels&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;percent_format&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;())&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_bar&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;stat&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;identity&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_hline&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;yintercept&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;linetype&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;dashed&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;coord_flip&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ylab&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Democrat/Republican share of page posts&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;xlab&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;scale_fill_manual&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;values&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;blue&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;red&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;legend.position&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;none&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Political Issues Discussed by Party\n&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</description>
    </item>
    
    <item>
      <title>Selective exposure in the age of social media: Endorsements trump partisan source affiliation when selecting news online</title>
      <link>https://solmessing.netlify.app/publication/messing-2014-selective/</link>
      <pubDate>Wed, 01 Jan 2014 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/messing-2014-selective/</guid>
      <description></description>
    </item>
    
    <item>
      <title>The Impression of Influence: Legislator Communication, Representation, and Democratic Accountability</title>
      <link>https://solmessing.netlify.app/publication/gri-mes-wes-14/</link>
      <pubDate>Wed, 01 Jan 2014 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/gri-mes-wes-14/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;Media: &lt;a href=&#34;http://www.mischiefsoffaction.com/2015/01/its-frequency-not-size-compromise.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mischiefs of Faction&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Do attitudes about immigration predict willingness to admit individual immigrants? A cross-national test of the person-positivity bias</title>
      <link>https://solmessing.netlify.app/publication/iyengar-2013-attitudes/</link>
      <pubDate>Tue, 01 Jan 2013 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/iyengar-2013-attitudes/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Evaluation of the novel USPIO GEH121333 for MR imaging of cancer immune responses</title>
      <link>https://solmessing.netlify.app/publication/shi-2013-evaluation/</link>
      <pubDate>Tue, 01 Jan 2013 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/shi-2013-evaluation/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Ferumoxytol: a new, clinically applicable label for stem-cell tracking in arthritic joints with MRI</title>
      <link>https://solmessing.netlify.app/publication/khurana-2013-ferumoxytol/</link>
      <pubDate>Tue, 01 Jan 2013 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/khurana-2013-ferumoxytol/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Friends that Matter: How Social News Shapes Political Knowledge, Attitudes, and Behavior</title>
      <link>https://solmessing.netlify.app/publication/messing-2013-friends/</link>
      <pubDate>Tue, 01 Jan 2013 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/messing-2013-friends/</guid>
      <description>&lt;p&gt;&lt;a href=&#34;https://www.dropbox.com/s/n0x4iepyj9pzwiw/CH7brief.pdf?raw=true&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://www.dropbox.com/s/n0x4iepyj9pzwiw/CH7brief.pdf?raw=true&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Iron administration before stem cell harvest enables MR imaging tracking after transplantation</title>
      <link>https://solmessing.netlify.app/publication/khurana-2013-iron/</link>
      <pubDate>Tue, 01 Jan 2013 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/khurana-2013-iron/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Role of diffusion-weighted imaging in differentiating benign and malignant pediatric abdominal tumors</title>
      <link>https://solmessing.netlify.app/publication/gawande-2013-role/</link>
      <pubDate>Tue, 01 Jan 2013 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/gawande-2013-role/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Insight From Cleveland And Tufte On Plotting Numeric Data By Groups</title>
      <link>https://solmessing.netlify.app/post/visualization-series-insight-from-cleveland-and-tufte-on-plotting-numeric-data-by-groups/</link>
      <pubDate>Sun, 04 Mar 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/visualization-series-insight-from-cleveland-and-tufte-on-plotting-numeric-data-by-groups/</guid>
      <description>&lt;p&gt;After my post on making &lt;a href=&#34;http://solomonmessing.wordpress.com/2011/11/26/putting-it-all-together-concise-code-to-make-dotplots-with-weighted-bootstrapped-standard-errors/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;dotplots with concise code using plyr and ggplot&lt;/a&gt;, I got an email from my dad who practices immigration law and runs a &lt;a href=&#34;http://www.messinglawoffices.com/default.aspx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;website with a variety of immigration resources and tools&lt;/a&gt;.  He pointed out that the post was written for folks who already know that they want to make dot plots, and who already know about bootstrapped standard errors.  That&amp;rsquo;s not many people.&lt;/p&gt;
&lt;p&gt;In an attempt to appeal to a broader audience, I&amp;rsquo;m starting a series in which I&amp;rsquo;ll outline the key principles I use when developing a visualization.  In this post, I&amp;rsquo;ll articulate these principles, which combine some of Tuft&amp;rsquo;s aesthetic guidelines with Cleveland&amp;rsquo;s scientific approach to visualization, which is based on the psychological processes involved in making sense of visualizations, and has been rigorously tested via randomized controlled experiments.  Based on these principles, I&amp;rsquo;ll argue that dotplots and scatterplots are better than other types of plots (especially pie charts) in most situations.  In later posts, I&amp;rsquo;ll demonstrate another innovation whose widespread use I&amp;rsquo;ll credit to Cleveland and Tufte: the use of multiple panels (aka small multiples, trellis graphics, facets, generalized draftsman&amp;rsquo;s displays, multivar charts) to clearly convey the same information embedded in more complex and difficult to read visualizations, including multiple line plots and mosaic plots. In future posts I&amp;rsquo;ll also emphasize why it is important to provide some indication of the noise present in the underlying data using error bars or bands.  Along the way, I&amp;rsquo;ll put you to the test&amp;ndash;I&amp;rsquo;ll present some visualizations of the same data using different visualization techniques and ask you to try to get as much information as you can in 2 seconds from each type of visualization.&lt;/p&gt;
&lt;p&gt;A good visualization conveys key information to those who may have trouble interpreting numbers and/or statistics, which can make your findings accessible to a wider audience (more on this below).  Visualizations also give your audience a break from lexical processing, which is especially useful when you are presenting your findings&amp;ndash;people can listen to you and process the findings from a well-designed visual at the same time, but most people have trouble listening while reading your PowerPoint bullet points.  Visualizations also convey key information embedded in massive amounts of data, which can aid your own exploratory analysis of data, no matter how massive.&lt;/p&gt;
&lt;p&gt;Yet most visualizations are flawed, drawn using elements that make it unnecessarily difficult for the human visual system to make sense of things.  I see a lot of these visualizations attending research presentations, screening incoming draft manuscripts as the assistant editor for &lt;a href=&#34;http://www.tandf.co.uk/journals/upcp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Political Communication&lt;/a&gt;, and as a consumer of media info-graphics (CNN is especially bad, have a look at &lt;a href=&#34;http://tech.fortune.cnn.com/tag/pie-chart/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this monstrosity&lt;/a&gt;).  Kevin Fox has an &lt;a href=&#34;http://fury.com/2010/03/why-3d-pie-charts-are-bad/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;especially compelling visual speaking to this here&lt;/a&gt;. A big part of the problem is that Microsoft makes it easy to draw flashy but ultimately confusing visualizations in Excel.  If you are too busy to read this post in full, follow this short list of guidelines and you&amp;rsquo;ll be on your way to producing elegant visualizations that impose a minimal cognitive burden on your audience:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Never represent something in 2 or &lt;a href=&#34;http://www.psdgraphics.com/wp-content/uploads/2009/02/3d_pie_chart.jpg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;worse yet 3 dimensions&lt;/a&gt; if it can be represented in one&amp;mdash;NEVER use pie charts, 3-D pie charts, stacked bar charts, or 3-D bar charts.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Remove as much chart junk as possible&amp;ndash;unnecessary gridlines, shading, borders, etc.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Give your audience a sense of the noise present in your data&amp;ndash;draw error bars or confidence bands if you are plotting estimates.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If you want to plot multiple types of groups on a single outcome (the visual analog of cross-tabulations/marginals), use &lt;a href=&#34;http://solomonmessing.files.wordpress.com/2011/11/trtbypid2.png?w=640&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;multi-paneled plots&lt;/a&gt;. These can also help if &lt;a href=&#34;http://www.bo.astro.it/~eps/buz10503/ff08.jpg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;overploting looks too cluttered&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Avoid mosaic plots. Instead use &lt;a href=&#34;http://wiki.stdout.org/rcookbook/Graphs/Facets%20%28ggplot2%29?action=AttachFile&amp;amp;do=get&amp;amp;target=hp_sex_smoker_free_free.png&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paneled histograms&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Ditch the legend if you can (you almost always can).&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The rest of the content in this series emphasizes why it makes sense to follow these guidelines. In this post I&amp;rsquo;ll look at the first point in detail and touch on the sixth. These two guidelines are most relevant when you want to look at a quantitative variable (e.g., earnings, vote-share, temperature, etc.) across different qualitative groupings (e.g., industry segment, candidate, party, racial group, season, etc.).  This is one of the most common visualization tasks in business, media, and social science, and for this task people often use pie charts and/or bar charts, and occasionally dot plots.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The science of graphical perception&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;When most people think about visualization, they think first of &lt;a href=&#34;http://www.edwardtufte.com/tufte/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Edward Tufte&lt;/a&gt;.  Tufte emphasizes integrity to the data, showing relationships between phenomena, and above all else aesthetic minimalism.  I appreciate his ruthless crusade against &lt;a href=&#34;http://chartjun%20k.karmanaut.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;chart junk&lt;/a&gt; and &lt;a href=&#34;http://jakeporway.com/2011/08/data-without-borders-logo-contest/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;pie charts (nice quote from Data without Borders)&lt;/a&gt;. We share an affinity for multipanel plotting approaches, which he calls &amp;ldquo;small multiples,&amp;rdquo; (thanks to &lt;a href=&#34;http://www.stanford.edu/~rjweiss/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Rebecca Weiss&lt;/a&gt; for pointing this out) though I think people give Tufte too much credit for their invention&amp;mdash;both &lt;a href=&#34;http://www.juiceanalytics.com/writing/better-know-visualization-small-multiples/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;juiceanalytics&lt;/a&gt; and &lt;a href=&#34;http://www.infovis-wiki.net/index.php/Small_Multiples&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;infovis-wiki&lt;/a&gt; write that Cleveland introduced the concept/principle. However, both Cleveland and Tufte published books in 1983 discussing the use of multipanel displays; &lt;a href=&#34;http://blog.revolutionanalytics.com/2011/11/small-multiples-of-the-sky.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;David Smith over at Revolutions&lt;/a&gt; writes that &amp;ldquo;the &amp;ldquo;small-multiples&amp;rdquo; principle of data visualization [was] pioneered by Cleveland and popularized in Tufte&amp;rsquo;s first book&amp;rdquo;; and the earliest reference to a work containing multipanel displays I could find was published *long* before Tufte&amp;rsquo;s 1983 work&amp;ndash;Seder, Leonard (1950), &amp;ldquo;Diagnosis with Diagrams—Part I&amp;rdquo;, Industrial Quality Control (New York, New York: American Society for Quality Control) 7 (1): 11–19.&lt;/p&gt;
&lt;iframe src=&#34;https://giphy.com/embed/11JbaLzOXsg6Fq&#34; width=&#34;200&#34; frameBorder=&#34;0&#34; class=&#34;giphy-embed&#34; align=&#34;right&#34;&gt;&lt;/iframe&gt;
I&#39;m less sure about Tufte&#39;s advice to always show axes starting at zero, which can make comparison between two groups difficult, and to &#34;show causality,&#34; which can end up misleading your readers.  Of course, the visualizations on display in the glossy pages of Tufte&#39;s books are beautiful.  But while his books are full of general advice that we should all keep in mind when creating plots, he does not put forth a theory of what works and what doesn&#39;t when trying to visualize data. 
&lt;p&gt;Cleveland (with Robert McGill) develops such a theory and subjects it to rigorous scientific testing. In my last post I linked to one of Cleveland&amp;rsquo;s studies showing that &lt;a href=&#34;https://www.cs.ubc.ca/~tmm/courses/cpsc533c-04-spr/readings/cleveland.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;dots (or bars) aligned on the same scale are indeed the best visualization to convey a series of numerical estimates&lt;/a&gt;.  In this work, Cleveland examined how accurately our visual system can process visual elements or &amp;ldquo;perceptual units&amp;rdquo; representing underlying data.  These elements include markers aligned on the same scale (e.g., dot plots, scatterplots, ordinary bar charts), the length of lines that are not aligned on the same scale (e.g., stacked bar plots), area (pie charts and mosaic plots), angles (also pie charts), shading/color, volume, curvature, and direction.&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-graphical-elements-cleveland&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/graphicalelementscleveland.png&#34; alt=&#34;Graphical Elements (Cleveland)&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Graphical Elements (Cleveland)
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;He runs two experiments: the first compares judgements about relative position (grouped bar charts) to judgements based only on length (stacked bar charts); the second compares judgements about relative position (ordinary bar charts) to judgements about angles/area (pie charts).  Here are the materials he uses, courtesy of the &lt;a href=&#34;http://graphics.stanford.edu/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Stanford Computer Graphics Lab&lt;/a&gt;:&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-graphical-perception-experiments&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/slide012.png&#34; alt=&#34;Graphical perception experiments&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Graphical perception experiments
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;















&lt;figure  id=&#34;figure-graphical-perception-experiments&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/slide013.png&#34; alt=&#34;Graphical perception experiments&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      Graphical perception experiments
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;The results are resoundingly clear&amp;mdash;judgements about position relative to a baseline are dramatically more accurate than judgements about angles, area, or length (with no baseline).  Hence, he suggests that we replace pie charts with bar charts or dot plots and that we substitute stacked bar charts for grouped bar charts.&lt;/p&gt;
&lt;p&gt;A striking and often overlooked finding in this work is the fact that the group of participants without technical training, &amp;ldquo;mostly ordinary housewives&amp;rdquo; as Cleveland describes them, performed &lt;em&gt;just as well&lt;/em&gt; as the group of mostly men with substantial technical training and experience.   This finding provides evidence for something that I&amp;rsquo;ve long suspected: that visualizations make it easier for people lacking quantitative experience to understand your results, serving to level the playing field.  If you want your findings to be broadly accessible, it&amp;rsquo;s probably better to present a visualization rather than a bunch of numbers.  It also suggests that if someone is having trouble interpreting your visualizations, it&amp;rsquo;s probably your fault.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Dotplots versus pie charts and stacked barplots&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Now let&amp;rsquo;s put this to the test.  Take a look at each visualization below for two seconds, looking for the percent of the vote that Mitt Romney, Ron Paul, and Jon Huntsman got.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://solmessing.netlify.app/img/primarydot2.png&#34;&gt;















&lt;figure  id=&#34;figure-primarydot&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/primarydot2.png&#34; alt=&#34;primaryDot&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      primaryDot
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;a href=&#34;https://solmessing.netlify.app/img/primarypie1.png&#34;&gt;















&lt;figure  id=&#34;figure-primarypie&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/primarypie1.png&#34; alt=&#34;primaryPie&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      primaryPie
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;a href=&#34;https://solmessing.netlify.app/img/primarystacked.png&#34;&gt;















&lt;figure  id=&#34;figure-primarystacked&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/primarystacked.png&#34; alt=&#34;primaryStacked&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      primaryStacked
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Which is easiest to read? Which conveys information most accurately? Let&amp;rsquo;s first take a look at the most critical information&amp;ndash;the order in which the candidates placed.  In all plots, the candidates are arrayed in order from highest to least vote share, and it&amp;rsquo;s easy to see that Mitt won.  But once we start looking at who came in second, third, and so on, differences emerge.  It&amp;rsquo;s slightly harder to process order in the pie chart because your eye has to go around the plot rather than up and down in a straight line.  In the stacked bar chart, we need to look up which color corresponds to which candidate&amp;rsquo;s in the legend (as Tufte told us not to use), adding a layer of cognitive processing.&lt;/p&gt;
&lt;p&gt;Second, which conveys estimates most accurately? The dot plot is the clear winner here.  We can quickly see that Romney got about 37%, Paul got about 24%, and Huntsman got about 16%, just by looking at dots relative to the axis.  When we look at the pie chart, it&amp;rsquo;s really tough to estimate the exact percent each candidate got.  Same with the stacked bar chart. We could add numbers to the pie and bar charts, which would even things out to some extent, but then why not just display a table with exact percents?&lt;/p&gt;
&lt;p&gt;One argument I used to hear all the time when I worked in industry is that pie charts &amp;ldquo;convey a sense of proportion.&amp;rdquo;  Well, sure, I guess I can kind of guestimate that Ron Paul&amp;rsquo;s vote share is about 1/4.  What about Jon Huntsman? Hmm, it looks like about 15 percent, which is 3/20.  But wait, why do I want to convert things into fractions anyway? I don&amp;rsquo;t think in terms of fractions, I think in terms of percents.  And if I really care about proportion, I suppose I could extend the axis from 0 to 100.&lt;/p&gt;
&lt;p&gt;Suppose I want to plot results for the top 15 candidates, not just the top 6?  Here&amp;rsquo;s what happens:&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://solmessing.netlify.app/img/primarydot15.png&#34;&gt;















&lt;figure  id=&#34;figure-primarydot15&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/primarydot15.png&#34; alt=&#34;primaryDot15&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      primaryDot15
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;a href=&#34;https://solmessing.netlify.app/img/primarypie15.png&#34;&gt;















&lt;figure  id=&#34;figure-primarypie15&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/primarypie15.png&#34; alt=&#34;primaryPie15&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      primaryPie15
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;No contest, the pie chart fails completely.  We&amp;rsquo;d need to add a legend with colors for each candidate, which adds another layer of cognitive processing&amp;ndash;we&amp;rsquo;d need to look up each color in the lengend as we go.  And even after adding the legend, you wouldn&amp;rsquo;t be able to distinguish the lower performing candidates from say write-in votes because the pie slices would be too small.  The stacked bar chart will fail for the same reasons, so I&amp;rsquo;ve excluded it in the interest of brevity.  Note that we don&amp;rsquo;t need to add colors to the dotplot to convey the same information, which saves an extra plotting element that we can use to represent something else (say candidate&amp;rsquo;s campaign funds or total assets).  And, on top of it all, the dot plot takes up less screen/page real estate!&lt;/p&gt;
&lt;p&gt;Why do I use dot plots instead of ordinary bar charts? A &lt;a href=&#34;http://www.perceptualedge.com/articles/b-eye/encoding_values_in_graph.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;nice visualization guide from perceptualedge.com&lt;/a&gt; points out that often we want to only visualize differences between groups in a narrow range (they use an example wherein monthly expenses vary from $4,250-$5,500). But the length of a bar is supposed to facilitate accurate comparisons between values, so when you use a bar plot starting from $4,250, the length between bars dramatically exaggerates the actual differences. Dot plots do not have this problem because dot encode values using only location, so one must reference the axis to interpret the value.&lt;/p&gt;
&lt;p&gt;A related points is that bars are often used to convey counts&amp;ndash;we use them in histograms to represent frequency and track say counts of dollars earned/raised in bar charts.  In fact, a team of doctors I work with at the med school recently sent in a manuscript to Radiology containing a bar chart plotting mean values between groups; they got back the following comment from the statistical reviewer: &amp;ldquo;the y-axis is quantitative but the data are represented using bars as if the data were counts.&amp;rdquo;  People often use bar plots to convey estimates of means (and &lt;a href=&#34;http://www.stanford.edu/~messing/APSAPoster.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;I&amp;rsquo;ve certainly done this&lt;/a&gt;), which can serve to exaggerate differences in means and hence effect sizes if you do not plot the bars from zero.&lt;/p&gt;
&lt;p&gt;In addition, dot plots have aesthetic advantages.  They convey the numerical estimate in question with a single one-dimensional point, rather than a two dimensional bar.  There&amp;rsquo;s simply less that the eye needs to process.  Accordingly, if a pattern across qualitative groupings exists, it&amp;rsquo;s often easier to see with a dot plot.  For example, below I plot the average user ratings for each article to which &lt;a href=&#34;http://www.stanford.edu/~seanjw/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Sean Westwood&lt;/a&gt; and I exposed subjects in a news reading experiment.  The pattern that emerges is an &amp;ldquo;S&amp;rdquo; curve in which one or two stories dominate the ratings, most are sort of average, and a few are uniformly terrible. Note that you&amp;rsquo;d probably want to use something like this more for yourself than to communicate your results to others as it might overload your audience with too much information&amp;ndash;you&amp;rsquo;d do better to select a subset of these articles or remove some of the ones in the middle (thanks to Yph Lelkes for making this point).&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://solmessing.netlify.app/img/dotplot-story-rating.png&#34;&gt;















&lt;figure  id=&#34;figure-dotplot-story-rating&#34;&gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/dotplot-story-rating.png&#34; alt=&#34;dotplot-story-rating&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;figcaption&gt;
      dotplot-story-rating
    &lt;/figcaption&gt;&lt;/figure&gt;
&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;One question that remains is if pie charts are so bad, why are they so common? Perhaps we like them because we find them comforting just as we find pies and pizza? Well if so we&amp;rsquo;d expect pie charts to be less common in places like Japan and China where people grow up eating different food.  Consider info-graphics in newspapers: I haven&amp;rsquo;t yet done a systematic content analysis, but I was unable to find a single pie chart in Japan&amp;rsquo;s Yomimuri Shimbun nor the Asahi Shimbun; nor in China&amp;rsquo;s Beijing Daily nor Sing Tao Daily.  I did see plenty of maps, however, which I suppose one could argue are reminiscent of noodles.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Implementation&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The most efficient way to produce solid visualizations with the ability to implement multiple panels, proper standard error estimates, and dot plots is probably in R using the ggplot2 package.  If you do not have time to learn R and remain tied to MS-Excel stick to ordinary barplots to visualize quantitative variables among multiple groups (not recommended).&lt;/p&gt;
&lt;p&gt;Otherwise, if you don&amp;rsquo;t already use it, &lt;a href=&#34;http://cran.r-project.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;download R&lt;/a&gt; and a decent editor like &lt;a href=&#34;http://rstudio.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Rstudio&lt;/a&gt;.  Then get started with ggplot2 and dot plots by running the following code chunk which will replicate the election figure above:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;read.csv&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;https://SolomonMg.github.io/img/primaryres.csv&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;as.is&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# sort data in order of percent of vote:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;[order&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Percentage&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;decreasing&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# only show top 15 candidates:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;pres[1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;15&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# create a precentage variable&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Percentage&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Percentage&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;100&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# reorder the Candidate factor by percentage for plotting purposes:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Candidate&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;reorder&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Candidate&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Percentage&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# To install ggplot2, run the following line after deleting the #&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#34;ggplot2&amp;#34;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ggplot2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;pres&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Percentage&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;factor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Candidate&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_point&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;xlab&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;Percent of Vote&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;ylab&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;Candidate&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;New Hampshire Primary 2012&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;After loading our data and running a few preliminary data processing operations, we pass ggplot our data set, &amp;ldquo;pres,&amp;rdquo; then we tell it what aesthetic elements we want to use, in this case that x is going to be our &amp;ldquo;Percentage&amp;rdquo; variable and y is going to be our &amp;ldquo;Candidate&amp;rdquo; variable. We tell ggplot that we want to display points for every xy pair. We also tell it to use the black and white theme, and pass some obscure axis options that ensures the axis plot correctly. Then we tell it what to label the x and y axis, and give it a title.&lt;/p&gt;
&lt;p&gt;We can also reproduce the article ratings by story plot above using ggplot2 (even though I originally produced the plot using the lattice package).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# To install ggplot2, run the following line after deleting the #&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#install.packages(&amp;#34;ggplot2&amp;#34;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ggplot2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;load&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;file&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;https://SolomonMg.github.io/img/db.Rda&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# if you haven&amp;#39;t installed dplyr, delete the # and run this line:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# install.packages(&amp;#34;dplyr&amp;#34;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;dplyr&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;table&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;db&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;story&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# first we use plyr to calculate the mean rating and SE for each story&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ratingdat&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;db&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;group_by&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;story&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%&amp;gt;%&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;summarise&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;M&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;mean&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rating&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;na.rm&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;SE&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;sd&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rating&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;na.rm&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;sqrt&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;length&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;na.omit&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rating&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;N&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;length&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;na.omit&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rating&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# make story into an ordered factor, ordering by mean rating:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ratingdat&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;story&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;factor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ratingdat&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;story&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ratingdat&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;story&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;reorder&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ratingdat&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;story&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;ratingdat&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;M&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# take a look at our handiwork:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ratingdat&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;M&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;xmin&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;M&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;SE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;xmax&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;M&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;SE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;story&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;geom_point&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;geom_segment&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;aes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;M&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;SE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;xend&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;M&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;SE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;y&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;story&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;yend&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;story&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;theme_bw&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;xlab&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;Mean rating&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;ylab&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;Story&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggtitle&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;Rating article by Story, with SE&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Now save&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;ggsave&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;file&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#34;plots/dotplot-story-rating.pdf&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;height&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;14&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;width&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;8.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</description>
    </item>
    
    <item>
      <title>Working with Bipartite/Affiliation Network Data in R</title>
      <link>https://solmessing.netlify.app/post/working-with-bipartite-affiliation-network-data-in-r/</link>
      <pubDate>Sun, 04 Mar 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/post/working-with-bipartite-affiliation-network-data-in-r/</guid>
      <description>&lt;p&gt;Data can often be usefully conceptualized in terms affiliations between people (or other key data entities). It might be useful analyze common group membership, common purchasing decisions, or common patterns of behavior. This post introduces bipartite/affiliation network data and provides R code to help you process and visualize this kind of data. I recently updated this for use with larger data sets, though I put it together a while back.&lt;/p&gt;
&lt;h2 id=&#34;preliminaries&#34;&gt;Preliminaries&lt;/h2&gt;
&lt;p&gt;Much of the material here is covered in the more comprehensive &lt;a href=&#34;http://sna.stanford.edu/rlabs.php&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;Social Network Analysis Labs in R and SoNIA,&amp;rdquo;&lt;/a&gt; on which I collaborated with Dan McFarland, Sean Westwood and Mike Nowak.
For a great online introduction to social network analysis see the online book &lt;a href=&#34;http://www.faculty.ucr.edu/~hanneman/nettext/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Introduction to Social Network Methods&lt;/a&gt; by Robert Hanneman and Mark Riddle.&lt;/p&gt;
&lt;h2 id=&#34;bipartiteaffiliation-network-data&#34;&gt;Bipartite/Affiliation Network Data&lt;/h2&gt;
&lt;p&gt;A network can consist of different &amp;lsquo;classes&amp;rsquo; of nodes. For example, a two-mode network might consist of people (the first mode) and groups in which they are members (the second mode). Another very common example of two-mode network data consists of users on a particular website who communicate in the same forum thread.
Here&amp;rsquo;s a short example of this kind of data. Run this in R for yourself - just copy an paste into the command line or into a script and it will generate a dataframe that we can use for illustrative purposes:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;data.frame&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;person&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Sam&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Sam&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Sam&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Greg&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Tom&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Tom&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Tom&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Mary&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Mary&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;group&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;a&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;b&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;c&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;a&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;b&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;c&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;d&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;b&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;d&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;n&#34;&gt;stringsAsFactors&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;bp&#34;&gt;F&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;person&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;group&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;    &lt;span class=&#34;n&#34;&gt;Sam&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;a&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;    &lt;span class=&#34;n&#34;&gt;Sam&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;b&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;3&lt;/span&gt;    &lt;span class=&#34;n&#34;&gt;Sam&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;c&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;4&lt;/span&gt;   &lt;span class=&#34;n&#34;&gt;Greg&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;a&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;5&lt;/span&gt;    &lt;span class=&#34;n&#34;&gt;Tom&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;b&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;6&lt;/span&gt;    &lt;span class=&#34;n&#34;&gt;Tom&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;c&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;7&lt;/span&gt;    &lt;span class=&#34;n&#34;&gt;Tom&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;d&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;8&lt;/span&gt;   &lt;span class=&#34;n&#34;&gt;Mary&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;b&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;m&#34;&gt;9&lt;/span&gt;   &lt;span class=&#34;n&#34;&gt;Mary&lt;/span&gt;     &lt;span class=&#34;n&#34;&gt;d&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;c1&#34;&gt;Fast, efficient two-mode to one-mode conversion in R&lt;/h2&gt;
&lt;p&gt;Suppose we wish to analyze or visualize how the people are connected directly - that is, what if we want the network of people where a tie between two people is present if they are both members of the same group? We need to perform a two-mode to one-mode conversion.&lt;/p&gt;
&lt;p&gt;To convert a two-mode incidence matrix to a one-mode adjacency matrix, one can simply multiply an incidence matrix by its transpose, which sum the common 1&amp;rsquo;s between rows. Recall that matrix multiplication entails multiplying the k-th entry of a row in the first matrix by the k-th entry of a column in the second matrix, then summing, such that the ij-th row-column entry in resulting matrix represents the dot-product of the i-th row of the first matrix and the j-th column of the second. In mathematical notation:&lt;/p&gt;
&lt;p&gt;$$
AB = \left [
\begin{array}{cc}
a &amp;amp; b  \\\
c &amp;amp; d
\end{array} \right ]
\left [ \begin{array}{cc}
e &amp;amp; f  \\\
g &amp;amp; h
\end{array} \right ] =
\left [
\begin{array}{cc}
ae+bg &amp;amp; af+bh \\\
ce+dg &amp;amp; cf+dh
\end{array}
\right ]
$$&lt;/p&gt;
&lt;p&gt;Notice further that multiplying a matrix by its transpose yields the following:&lt;/p&gt;
&lt;p&gt;$$
\begin{align}
AA&amp;rsquo; =
\left[
\begin{array}{cc}
a &amp;amp; b  \\\
c &amp;amp; d
\end{array}
\right]
\left[
\begin{array}{cc}
a &amp;amp; c  \\\
b &amp;amp; d
\end{array}
\right] =
\left[
\begin{array}{cc}
aa+bb &amp;amp; ac+bd \\\
ca+db &amp;amp; cc+dd
\end{array}
\right]
\end{align}
$$&lt;/p&gt;
&lt;p&gt;Because our incidence matrix consists of 0&amp;rsquo;s and 1&amp;rsquo;s, the off-diagonal entries represent the total number of common columns, which is exactly what we wanted. We&amp;rsquo;ll use the &lt;code&gt;%*%&lt;/code&gt; operator to tell R to do exactly this. Let&amp;rsquo;s take a look at a small example using toy data of people and groups to which they belong. We&amp;rsquo;ll coerce the data to an incidence matrix, then multiply the incidence matrix by its transpose to get the number of common groups between people.&lt;/p&gt;
&lt;p&gt;This is easy to do using the matrix algebra functions included in R. But first, you need to restructure your (edgelist) network data as an incidence matrix. An incidence will record a 1 for row-column combinations where a tie is present and 0 otherwise. One easy way to do this in R is to use the table function and then coerce the table object to a matrix object:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;m&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;table&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;df&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;M&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;as.matrix&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;m&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If you are using the network or sna packages, a network object be coerced via &lt;code&gt;as.matrix(your-network)&lt;/code&gt;; with the igraph package use &lt;code&gt;get.adjacency(your-network)&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;This is great, but what about if we are working with a really large data set? Network data is almost always sparse&amp;mdash;there are far more pairwise combinations of potential connections than actual observed connections. Hence, we&amp;rsquo;d actually prefer to keep the underlying data structured in edgelist format, but we&amp;rsquo;d also like access to R&amp;rsquo;s matrix algebra functionality.&lt;/p&gt;
&lt;p&gt;We can get the best of both worlds using the Matrix library to construct a sparse triplet representation of a matrix. But we&amp;rsquo;d also like to avoid building the entire incidence matrix and just feed Matrix our edgelist directly, a point that came up in a recent conversation I had with &lt;a href=&#34;http://seanjtaylor.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Sean Taylor&lt;/a&gt;. We feed &lt;code&gt;Matrix&lt;/code&gt; our &amp;lsquo;person&amp;rsquo; column to index &amp;lsquo;i&amp;rsquo; (rows in the new incidence matrix), our &amp;lsquo;group&amp;rsquo; column to index j (columns in the new incidence matrix), and we repeat &amp;lsquo;1&amp;rsquo; for the length of the edgelist to denote an incidence.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;Matrix&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;spMatrix&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;nrow&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;length&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;unique&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;person&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ncol&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;length&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;unique&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;group&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;i&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;as.numeric&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;factor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;person&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;j&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;as.numeric&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;factor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;group&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;x&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rep&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;length&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;as.numeric&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;person&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)))&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;levels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;factor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;person&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;levels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;factor&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;df&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;group&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We will either convert to the &amp;lsquo;mode&amp;rsquo; represented by the columns or by the rows.
To get the one-mode representation of ties between rows (people in our example), multiply the matrix by its transpose. Note that you must use the matrix-multiplication operator &lt;code&gt;%*%&lt;/code&gt; rather than a simple astrisk. The R code is:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Arow&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;A&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%*%&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;But we can still do better! The function tcrossprod is faster and more efficient for this:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Arow&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;tcrossprod&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Arow will now represent the one-mode matrix formed by the row entities&amp;mdash;people will have ties to each other if they are in the same group, in our example. Here&amp;rsquo;s what it looks like:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Arow
4 x 4 sparse Matrix of class &amp;quot;dgCMatrix&amp;quot;
     Greg Mary Sam Tom
Greg    1    .   1   .
Mary    .    2   1   2
Sam     1    1   3   2
Tom     .    2   2   3
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To get the one-mode matrix formed by the column entities (i.e. the number of people) enter the following command:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Acol&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%*%&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Again, we can use tcrossprod to make this even more efficient:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;Acol&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;tcrossprod&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;A&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;And the resulting co-membership matrix is as follows:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Mcol
group
group a b c d
a 2 1 1 0
b 1 3 2 2
c 1 2 2 1
d 0 2 1 2
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Although we&amp;rsquo;ve used a very small network for our example, this code is highly extensible to the analysis of larger networks with R.&lt;/p&gt;
&lt;h2 id=&#34;c1&#34;&gt;Analysis of Two Mode Data and Mobility&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s work with some actual affiliation data, collected by Dan McFarland on student extracurricular affiliations. It&amp;rsquo;s a longitudinal data set, with 3 waves - 1996, 1997, 1998.  It consists of students (anonymized) and the student organizations in which they are members (e.g. National Honor Society, wrestling team, cheerleading squad, etc.).
What we&amp;rsquo;ll do is to read in the data, explore it, make a few two-to-one mode conversions, and visualize it.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Load the &amp;#39;igraph&amp;#39; library&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;library&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;igraph&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# (1) Read in the data files, NA data objects coded as &amp;#39;na&amp;#39;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magact96&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;read.delim&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;https://solomonmg.github.io/assets/img/mag_act96.txt&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;na.strings&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;na&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magact97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;read.delim&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;https://solomonmg.github.io/assets/img/mag_act97.txt&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;na.strings&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;na&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magact98&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;read.delim&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;https://solomonmg.github.io/assets/img/mag_act98.txt&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;na.strings&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;na&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Missing data is coded as &amp;ldquo;na&amp;rdquo; in this data, which is why we gave R the command na.strings = &amp;ldquo;na&amp;rdquo;.&lt;/p&gt;
&lt;p&gt;These files consist of four columns of individual-level attributes (ID, gender, grade, race), then a bunch of group membership dummy variables (coded &amp;ldquo;1&amp;rdquo; for membership, &amp;ldquo;0&amp;rdquo; for no membership).  We need to set aside the first four columns (which do not change from year to year).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magattrib&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;magact96[&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;as.matrix&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magact96[&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;);&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;magact96&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ID.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;as.matrix&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magact97[&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;);&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;magact97&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ID.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;as.matrix&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magact98[&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;);&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;magact98&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ID.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;By using the &lt;code&gt;[,-(1:4)]&lt;/code&gt; index, we drop those columns so that we have a square incidence matrix for each year, and then tell R to set the row names of the matrix to the student&amp;rsquo;s ID. Note that we need to keep the &amp;ldquo;.&amp;rdquo; after ID in this dataset (because it&amp;rsquo;s in the name of the variable).
Now we load these two-mode matrices into igraph:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;graph.incidence&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mode&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;all&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;i97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;graph.incidence&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mode&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;all&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;i98&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;graph.incidence&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mode&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;c&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;all&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;plotting-two-mode-networks&#34;&gt;Plotting two-mode networks&lt;/h3&gt;
&lt;p&gt;Now, let&amp;rsquo;s plot these graphs. The igraph package has excellent plotting functionality that allows you to assign visual attributes to igraph objects before you plot. The alternative is to pass 20 or so arguments to the &lt;code&gt;plot.igraph()&lt;/code&gt; function, which gets really messy.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s assign some attributes to our graph. First we set vertex attributes, making sure to make them slightly transparent by altering the gamma, using the &lt;code&gt;rgb(r,g,b,gamma)&lt;/code&gt; function to set the color. This makes it much easier to look at a really crowded graph, which might look like a giant hairball otherwise.
You can read up on the RGB color model &lt;a href=&#34;http://en.wikipedia.org/wiki/RGB_color_model&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Each node (or &amp;ldquo;vertex&amp;rdquo;) object is accessible by calling &lt;code&gt;V(g)&lt;/code&gt;, and you can call (or create) a node attribute by using the &lt;code&gt;$&lt;/code&gt; operator so that you call &lt;code&gt;V(g)$attribute&lt;/code&gt;. Here&amp;rsquo;s how to set the color attribute for a set of nodes in a graph object:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color[1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1295&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color[1296&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1386&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Notice that we index the &lt;code&gt;V(g)$color&lt;/code&gt; object by a seemingly arbitrary value, 1295.  This marks the end of the student nodes, and 1296 is the first group node. You can view which nodes are which by typing V(i96). R prints out a list of all the nodes in the graph, and those with a number are obviously different from those that consist of a group name.&lt;/p&gt;
&lt;p&gt;Now we&amp;rsquo;ll set some other graph attributes:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label.color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label.cex&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.4&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;6&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;frame.color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;NA&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;You can also set edge attributes. Here we&amp;rsquo;ll make the edges nearly transparent and slightly yellow because there will be so many edges in this graph:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now, we&amp;rsquo;ll open a pdf &amp;ldquo;device&amp;rdquo; on which to plot. This is just a connection to a pdf file. Note that the code below will take a minute or two to execute (or longer if you have a pre- Intel dual-core processor).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;pdf&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;i96.pdf&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout.fruchterman.reingold&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;dev.off&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that we&amp;rsquo;ve used the Fruchterman-Reingold force-directed layout algorithm here.  Generally speaking, the when you have a ton of edges, the Kamada-Kawai layout algorithm works well but, it can get really slow for networks with a lot of nodes. Also, for larger networks, layout.fruchterman.reingold.grid is faster, but can fail to produce a plot with any meaninful pattern if you have too many isolates, as is the case here. Experiment for yourself.
Here&amp;rsquo;s what we get:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/i96.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;It&amp;rsquo;s oddly reminiscent of a cresent and star, but impossible to read. Now, if you open the &lt;a href=&#34;https://solmessing.netlify.app/img/i96.pdf&#34;&gt;pdf output&lt;/a&gt;, you&amp;rsquo;ll notice that you can zoom in on any part of the graph ad infinitum without losing any resolution. How is that possible in such a small file? It&amp;rsquo;s possible because the pdf device output consists of data based on vectors: lines, polygons, circles, elipses, etc., each specified by a mathematical formula that your pdf program renders when you view it. Regular bitmap or jpeg picture output, on the other hand, consists of a pixel-coordinate mapping of the image in question, which is why you lose resolution when you zoom in on a digital photograph or a plot produced with most other programs.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s remove all of the isolates (the cresent), change a few aesthetic features, and replot. First, we&amp;rsquo;ll remove isloates, by deleting all nodes with a degree of 0, meaning that they have zero edges. Then, we&amp;rsquo;ll suppress labels for students and make their nodes smaller and more transparent. Then we&amp;rsquo;ll make the edges more narrow more transparent. Then, we&amp;rsquo;ll replot using various layout algorithms:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;delete.vertices&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;degree&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;==&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label[1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;857&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;NA&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color[1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;857&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt;  &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;size[1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;857&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;width&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.3&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;pdf&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;i96.2.pdf&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout.kamada.kawai&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;dev.off&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;pdf&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;i96.3.pdf&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout.fruchterman.reingold.grid&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;dev.off&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;pdf&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;i96.4.pdf&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout.fruchterman.reingold&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;dev.off&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;I personally prefer the Fruchterman-Reingold layout in this case. The nice thing about this layout is that it really emphasizes centrality&amp;ndash;the nodes that are most central are nearly always placed in the middle of the plot. Here&amp;rsquo;s what it looks like:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/i962.jpg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Very pretty, but you can&amp;rsquo;t see which groups are which at this resolution. Zoom assets/in on the &lt;a href=&#34;https://solmessing.netlify.app/img/samplepdf.pdf&#34;&gt;pdf output&lt;/a&gt;, and you can see things pretty clearly.&lt;/p&gt;
&lt;h3 id=&#34;two-mode-to-one-mode-data-transformation&#34;&gt;Two mode to one mode data transformation&lt;/h3&gt;
&lt;p&gt;We&amp;rsquo;ve emphasized groups in this visualization so much, that we might want to just create a network consisting of group co-membership. First we need to create a new network object. We&amp;rsquo;ll do that the same way for this network as for our example at the top of this page:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%*%&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g97e&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%*%&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g98e&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%*%&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;graph.adjacency&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mode&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;undirected&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now we need to tansform the graph so that multiple edges become an attribute ( &lt;code&gt;E(g)$weight&lt;/code&gt; ) of each unique edge:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;weight&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;count.multiple&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;simplify&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now we&amp;rsquo;ll set the other plotting parameters as we did above:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set vertex attributes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label.color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.8&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label.cex&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.6&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;6&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;frame.color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;NA&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set edge gamma according to edge weight&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;egam&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;weight&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;+.3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;max&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;log&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;weight&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;+.3&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;egam&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We set edge gamma as a function of how many edges exist between two nodes, or in this case, how many students each group has in common. For illustrative purposes, let&amp;rsquo;s compare how the Kamada-Kawai and Fruchterman-Reingold algorithms render this graph:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;pdf&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;i96e.pdf&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;main&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;layout.kamada.kawai&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout.kamada.kawai&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;i96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;main&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;s&#34;&gt;&amp;#39;layout.fruchterman.reingold&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout.fruchterman.reingold&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;dev.off&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;I like the Kamada-Kawai layout for this graph, because the center of the graph is too busy otherwise. And here&amp;rsquo;s what the resulting plot looks like:
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/i96e-kk.jpeg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;You can check out the difference between each layout yourself. Here&amp;rsquo;s what the &lt;a href=&#34;https://solmessing.netlify.app/img/i96e.pdf&#34;&gt;assets/pdf output looks like&lt;/a&gt;.  Page 1 shows the Kamada-Kawai layout and page 2 shows the Fruchterman Reingold layout.&lt;/p&gt;
&lt;h3 id=&#34;group-overlap-networks-and-plots&#34;&gt;Group overlap networks and plots&lt;/h3&gt;
&lt;p&gt;Now we might also be interested in the percent overlap between groups. Note that this will be a directed graph, because the percent overlap will not be symmetric across groups&amp;ndash;for example, it may be that 3/4 of Spanish NHS members are in NHS, but only 1/8 of NHS members are in the Spanish NHS. We&amp;rsquo;ll create this graph for all years in our data (though we could do it for one year only).
First we&amp;rsquo;ll need to create a percent overlap graph. We start by dividing each row by the diagonal (this is really easy in R):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ol96&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ol97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g97e&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ol98&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g98e&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Next, sum the matricies and set any NA cells (caused by dividing by zero in the step above) to zero:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magall&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;ol96&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;ol97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;ol98&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magall&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;[is.na&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magall&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that magall now consists of a percent overlap matrix, but because we&amp;rsquo;ve summed over 3 years, the maximun is now 3 instead of 1.
Let&amp;rsquo;s compute average club size, by taking the mean across each value in each diagonal:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magdiag&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;apply&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;cbind&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mean&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Finally, we&amp;rsquo;ll generate centrality measures for magall. When we create the igraph object from our matrix, we need to set weighted=T because otherwise igraph dichotomizes edges at 1. This can distort our centrality measures because now edges represent  more than binary connections&amp;ndash;they represent the percent of membership overlap.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magallg&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;graph.adjacency&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magall&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;weighted&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Degree&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallg&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;degree&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;degree&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallg&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Betweenness centrality&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallg&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;btwcnt&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;betweenness&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallg&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Before we plot this, we should probably filter some of the edges, otherwise our graph will probably be too busy to make sense of visually.  Take a look at the distribution of connection strength by plotting the density of the magall matrix:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;density&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magall&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/densitymagall.jpeg&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Nearly all of the edge weights are below 1&amp;ndash;or in other words, the percent overlap for most clubs is less than 1/3. Let&amp;rsquo;s filter at 1, so that an edge will consists of group overlap of more than 1/3 of the group&amp;rsquo;s members in question.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magallgt1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;magall&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magallgt1[magallgt1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;graph.adjacency&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallgt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;weighted&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;bp&#34;&gt;T&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Removes loops:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;simplify&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;remove.multiple&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;kc&#34;&gt;FALSE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;remove.loops&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;kc&#34;&gt;TRUE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Before we do anything else, we&amp;rsquo;ll create a custom layout based on Fruchterman.-Ringold wherein we adjust the coordates by hand using the tkplot gui tool to make sure all of the labels are visible. This is very useful if you want to create a really sharp-looking network visualization for publication.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;layout.fruchterman.reingold&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;tkplot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Let the plot load, then maximize the window, and select to View -&amp;gt; Fit to Screen so that you get maximum resolution for this large graph. Now hand-place the nodes, making sure no labels overlap:
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/tkplotscreenshot.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Pay special attention to whether the labels overlap (or might overlap if the font was bigger) along the vertical. Save the layout coordinates to the graph object:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;layout&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;tkplot.getcoords&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We use &amp;ldquo;1&amp;rdquo; here because only if this was the first tkplot object you called. If you called tkplot a few times, use the last plot object. You can tell which object is visible because at the top of the tkplot interface, you&amp;rsquo;ll see something like &amp;ldquo;Graph plot 1&amp;rdquo; or in the case of my screenshot above &amp;ldquo;Graph plot 7&amp;rdquo; (it was the seventh time I called tkplot).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set vertex attributes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label.color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.6&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;6&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;frame.color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;NA&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set edge attributes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;arrow.size&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.3&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set edge gamma according to edge weight&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;egam&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;weight&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;+.1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;max&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;weight&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;+.1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;color&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;rgb&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;.5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;egam&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;One thing that we can do with this graph is to set label size as a function of degree, which adds a &amp;ldquo;tag-cloud&amp;rdquo;-like element to the visualization:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;label.cex&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;degree&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;max&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;V&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;$&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;degree&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;m&#34;&gt;.3&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#note, unfortunately one must play with the formula above to get the&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;#ratio just right&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Let&amp;rsquo;s plot the results:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;pdf&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s&#34;&gt;&amp;#39;magallggt1customlayout.pdf&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;plot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magallggt1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;dev.off&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that we used the custom layout, which because we made part of the igraph object magallggt1, we did not need to specify in plot command.
assets/Here&amp;rsquo;s the &lt;a href=&#34;https://solmessing.netlify.app/img/magallggt1custom.pdf&#34;&gt;pdf output&lt;/a&gt;, and here&amp;rsquo;s what it looks like:&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;https://solmessing.netlify.app/img/magallggt1custom.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;This visualization reveals much more information about our network than our cresent-star visualization.&lt;/p&gt;
&lt;h3 id=&#34;mobility-markov-and-transition-probabilities&#34;&gt;Mobility, Markov, and Transition Probabilities&lt;/h3&gt;
&lt;p&gt;In order to shed light on how people flow through these groups, we&amp;rsquo;ll compute transition probabilities. These transition probabilities are more generally referred to as Markov chains.&lt;/p&gt;
&lt;p&gt;First we&amp;rsquo;ll create a new matrix that multiplies 1996 magnet with 1997 magnet so you see the number of students moving from 1996 membership to 1997 memberships.&lt;/p&gt;
&lt;p&gt;Before we actually do this, we need to do some data munging to make sure that the rows and columns for g96 and g97 are the same. We&amp;rsquo;ll use the match() function for this.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# First, let&amp;#39;s get an idea of how many column-names (activities) and row&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# names (student ids) are in common between the two years:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;intersect&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rnames&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;intersect&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Great, there are a lot of names in common. Now we&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# need to make sure we are only using the rows&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# and columns of each matrix that contain entries used in&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# both years. We also need to make sure that the columns and&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# rows are in the same order.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# In order to accomplish this we are going to exploit R&amp;#39;s&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# indexing capabilities. We are going to have R &amp;#34;rebuild&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# each matrix according to the order of rnames and cnames.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# We&amp;#39;ll use the match() function to accomplish this.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g96matched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g96[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g97matched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g97[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# We need to do the same thing for the diagonal of the matrix g96e, which is&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# our co-membership/affiliation matrix computed above:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;mag96diagmatched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g96e[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Now let&amp;#39;s check to make sure things worked correctly:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;which&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96matched&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;!=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97matched&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nf&#34;&gt;which&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96matched&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;!=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97matched&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now that these are effectively matricies, we can multiply to get the transition probability matrix:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;mag96_97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96matched&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%*%&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g97matched&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Let&amp;rsquo;s munge the 97 and 98 data and repeat:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;intersect&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;rnames&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;intersect&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g97matched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g97[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;g98matched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g98[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;rnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;row.names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;And again for the 97-98 transition:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;mag97_98&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;t&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97matched&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;%*%&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g98matched&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now we need to get the group-level membership matrix diagonal, ordered by the current set of columns.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;mag96diagmatched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g96e[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g96e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;mag97diagmatched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g97e[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g97e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;mag98diagmatched&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;diag&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;g98e[&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)),&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;nf&#34;&gt;match&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;cnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;colnames&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;g98e&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;And finally we can create the transition probability matrix! Divide magmob96_97 by mag96diagmatched in to get the transition probability matrix (Markov chain):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magmob96_97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mag96_97&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mag96diagmatched&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;magmob97_98&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;mag97_98&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mag97diagmatched&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now add the matrices and divide by 2:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;mobility_all&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;magmob96_97&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;+&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;magmob97_98&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now plot as with the event-overlap graphs!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Differentiation of Normal Thymus from Anterior Mediastinal Lymphoma and Lymphoma Recurrence at Pediatric PET/CT</title>
      <link>https://solmessing.netlify.app/publication/gawande-2012-differentiation/</link>
      <pubDate>Sun, 01 Jan 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/gawande-2012-differentiation/</guid>
      <description></description>
    </item>
    
    <item>
      <title>FDG PET/CT for the Evaluation of Normal Thymus, Lymphoma Recurrence, and Mediastinal Lymphoma in Pediatric Patients Response</title>
      <link>https://solmessing.netlify.app/publication/gawande-2012-fdg/</link>
      <pubDate>Sun, 01 Jan 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/gawande-2012-fdg/</guid>
      <description></description>
    </item>
    
    <item>
      <title>How Words and Money Cultivate a Personal Vote: The Effect of Legislator Credit Claiming on Constituent Credit Allocation</title>
      <link>https://solmessing.netlify.app/publication/grimmer-how-2012/</link>
      <pubDate>Sun, 01 Jan 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/grimmer-how-2012/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://solmessing.netlify.app/pdf/ccsup.pdf&#34;&gt;Supplementary Information&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Intravenous ferumoxytol allows noninvasive MR imaging monitoring of macrophage migration into stem cell transplants</title>
      <link>https://solmessing.netlify.app/publication/khurana-2012-intravenous/</link>
      <pubDate>Sun, 01 Jan 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/khurana-2012-intravenous/</guid>
      <description></description>
    </item>
    
    <item>
      <title>The 2012 Election Day Through the Facebook Lens</title>
      <link>https://solmessing.netlify.app/publication/messing-2012-ts/</link>
      <pubDate>Sun, 01 Jan 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/messing-2012-ts/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Who is a `deserving&#39;immigrant? An experimental study of Norwegian attitudes</title>
      <link>https://solmessing.netlify.app/publication/aalberg-2012-deserving/</link>
      <pubDate>Sun, 01 Jan 2012 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/aalberg-2012-deserving/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Measuring issue salience: Using supervised machine learning to generate data from free responses to the &#39;most important problem&#39; question</title>
      <link>https://solmessing.netlify.app/publication/messing-2011-measuring/</link>
      <pubDate>Sat, 01 Jan 2011 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/messing-2011-measuring/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Of course I wouldn&#39;t do that in real life: advancing the arguments for increasing realism in HCI experiments</title>
      <link>https://solmessing.netlify.app/publication/lew-2011-course/</link>
      <pubDate>Sat, 01 Jan 2011 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/lew-2011-course/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Do explicit racial cues influence candidate preference? The case of skin complexion in the 2008 campaign</title>
      <link>https://solmessing.netlify.app/publication/iyengar-2008-explicit/</link>
      <pubDate>Tue, 01 Jan 2008 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/publication/iyengar-2008-explicit/</guid>
      <description></description>
    </item>
    
    <item>
      <title></title>
      <link>https://solmessing.netlify.app/admin/config.yml</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/admin/config.yml</guid>
      <description></description>
    </item>
    
    <item>
      <title>Disclosures</title>
      <link>https://solmessing.netlify.app/disclosures/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/disclosures/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;SM is currently an employee of New York University.&lt;/li&gt;
&lt;li&gt;SM was previously an employee of Twitter and held a significant financial interest (SFI, defined as holding more than $5,000 in stock). SM&amp;rsquo;s employee equity was converted to cash following the conclusion of Twitter&amp;rsquo;s sale to Elon Musk. SM was against the company&amp;rsquo;s acquisition and asked to be included in the first round of layoffs. SM declined to accept severance compensation and did not sign the proposed severance agreement, which included a restrictive non-disparagement clause.&lt;/li&gt;
&lt;li&gt;SM was previously an employee of Facebook (now Meta) and at various points in time held a SFI in the company. SM sold all employee equity grants (RSUs) shortly after leaving the company in both 2015 and 2020.&lt;/li&gt;
&lt;li&gt;SM was previously an employee of Science Applications International Corporation and in the past held a SFI in the company.&lt;/li&gt;
&lt;li&gt;SM was previously an employee of Pew Research Center.&lt;/li&gt;
&lt;li&gt;SM was previously an employee of Acronym, including during the implementation of experiments designed to assess ad effectiveness.&lt;/li&gt;
&lt;li&gt;SM was previously a paid contractor for Morning Consult.&lt;/li&gt;
&lt;li&gt;SM has actively managed investments which may include SFIs in Apple, AMD, Alfabet (Google), Amazon, Intel, Meta, Microsoft, NVIDIA, Salesforce. SM has sometimes sold (ordinary) Facebook/Meta stock to avoid the appearance of a potential conflict of interest when conducting research relevant to the platform.&lt;/li&gt;
&lt;li&gt;SM has received scholarly funding from Google (now Alfabet).&lt;/li&gt;
&lt;li&gt;SM is an inventor on patents assigned to Twitter and Law-on-line.&lt;/li&gt;
&lt;li&gt;SM&amp;rsquo;s travel to or lodging at conferences has been supported by universities and past employers, including Pew Research Center (2015-2018), Facebook (2012-2020), Twitter (2021-2022).&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>MTB Gear</title>
      <link>https://solmessing.netlify.app/mtbgearweather/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/mtbgearweather/</guid>
      <description>&lt;p&gt;You need to dress approriately for mountainbiking/fat tire cycling in the colder temperatures of the upper midwest. Here&amp;rsquo;s what has worked for me after two winters outside of the Twin Cities in Minnesota:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;55-65: padded shorts, MTB shorts, knee pads, elbow pads. Alt: enduro shirt or long sleeve undershirt. Shoe covers. Full face helmet or skullcap.&lt;/li&gt;
&lt;li&gt;45-55: padded shorts, bike pants, long sleeve undershirt, short sleeve shirt, padded jacket or fleece mid layer. Vest or puffy jacket for nicer warmup. Shoe covers. Full face helmet or skullcap. Ski goggles.&lt;/li&gt;
&lt;li&gt;35-45: padded shorts, bike pants, long sleeve undershirt, padded jacket or fleece mid layer. Light outer layer w underarm zips. Shoe covers and toe caps, or legit boots. Ski/snowboard helmet + goggles. Warm gloves.&lt;/li&gt;
&lt;li&gt;25-35: padded shorts, bike pants, long sleeve undershirt, padded jacket or fleece mid layer. Light outer layer w underarm zips. Legit winter boots. Ski/snowboard helmet + goggles. Face mask. Handlebar mitts.&lt;/li&gt;
&lt;li&gt;15-25: padded shorts, ski pants, long sleeve undershirt, padded jacket or fleece mid layer. Light outer layer w underarm zips. Legit winter boots. Ski/snowboard helmet + goggles. Face mask. Handlebar mitts.&lt;/li&gt;
&lt;li&gt;Under 15: padded shorts, ski pants, long sleeve undershirt, padded jacket or fleece mid layer. Light outer layer w underarm zips. Cold weather winter boots (e.g., 45 NRTH Wolfgar), consider toe warmers. Ski/snowboard helmet + goggles. Face mask. Handlebar mitts w/ warm gloves.&lt;/li&gt;
&lt;/ol&gt;
</description>
    </item>
    
    <item>
      <title>Tech Stack</title>
      <link>https://solmessing.netlify.app/techsetup/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>https://solmessing.netlify.app/techsetup/</guid>
      <description>&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://gist.github.com/kevin-smets/8568070&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Best terminal (iterm 2) setup I&amp;rsquo;ve found yet&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://stackoverflow.com/a/42536153/34935&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Fix annoying error messages in RStudio&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.walware.de/goto/statet&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;StatET&lt;/a&gt;, an R plugin for Eclipse. Why not just RStudio? StatET has the best object browser available if you&amp;rsquo;re doing anything with JSON or lists of lists, and will not crash when R crashes, unlike RStudio. It also plays well with cloud instances. Though, yes, I find myself using RStudio on new machines because setting up StatET can be a challenge.&lt;/li&gt;
&lt;li&gt;Get started using &lt;a href=&#34;https://sourcethemes.com/academic/docs/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hugo - Academic&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Quickly snap windows side-by-side with shortcut keys on Mac using &lt;a href=&#34;https://github.com/rxhanson/Rectangle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Rectangle&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;How to setup &lt;a href=&#34;https://www.techrepublic.com/article/how-to-enable-custom-keyboard-shortcuts-in-gmail/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;shotcut keys&lt;/a&gt; in gmail.&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://sites.google.com/nyu.edu/nyu-hpc/hpc-systems/greene/software/open-ondemand-ood-with-condasingularity?authuser=0&amp;amp;pli=1&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Conda on Singularity&lt;/a&gt; (needed for NYU HPC)&lt;/li&gt;
&lt;li&gt;How to launch &lt;a href=&#34;https://stackoverflow.com/questions/63604427/launching-a-singularity-container-remotely-using-visual-studio-code&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Singularity container remotely in VSCode&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
  </channel>
</rss>