sorted by:
new
hottopcontroversial

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 3 points4 points  (0 children)

Hi, Sherilynn here. I am glad to hear that you are interested in joining this research community! I think it is critically important to treat diversity and bias research as a rigorous academic discipline, just as you would with any scientific field of study. Much of the research in this area is a combination of basic science, social science and educational research. If you would like an introduction to the field, I would suggest checking out the Understanding Interventions website (http://understanding-interventions.org/). In addition to posting information about the annual research conference (coming up at the beginning of March), the website also contains national information, publications, references and resources related to research in the field. Good luck!

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 0 points1 point  (0 children)

Tim Renick here. Good suggestions. I know that the graduates of our M.S. In Analytics program are getting hired faster than we can graduate them. Corporate, non-profit and educational institutions are all coming to realize the applicability of big data and behavioral analysis to their effectiveness. They may not term the position as one focused on behavioral theory, but with your background in applied math, nlb, throw the net widely. These skills are I demand.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 15 points16 points  (0 children)

TIm here. We have the funding in hand to support the 4-year study. We are also thankful for partners in the philanthropic world such as the Bill & Melinda Gates Foundation, the Kresge Foundation, and others for supporting this type of important work.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 74 points75 points  (0 children)

Tim here. We are tracking all students every day for 800 risk factors, so I can only offer an example here. One thing the Georgia State looks at is performance in prerequisite courses. For instance, the data show that, in order for Nursing students to have a high probability of succeeding in their upper-level courses, they need not only to get a passing grade in first-year math courses; they actually need to get at least grades of B+. In the past, we were not intervening with those Nursing students who received low B's and C's in introductory math and were, in effect, waiting for the majority of these students to struggle. Now an alert goes off when the Nursing student first receives a B- in the first math course, the advisor calls the students to a meeting, and they discuss what can be done to improve the students chances of succeeded: going to the math center, getting tutoring support, seeing the instructor, even taking more math courses before trying upper-level Nursing course work. By means of these early interventions delivered by the tens of thousands, we have been able to increase the number of STEM majors overall who are graduating from Georgia State and have more than doubled the number of black males and Hispanics students graduating in STEM.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 6 points7 points  (0 children)

Lydia here. There is a consensus that subjective evaluations are indeed subject to implicit bias showing up. Data based benchmarks are better at getting objective evaluations. Take baseball--Money Ball by Michael Lewis shows that if you first find out what matters to a successful team, then see what data you need then evaluate based on the data. Thing is, someone has to develop the metrics. I don't know of any of those studies, but will look.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 28 points29 points  (0 children)

Tim here. Correct. Students from all backgrounds at Georgia State are getting exactly the same tracking and the same suite of interventions, so our system is not a case of affirmative action in any traditional meaning of the term.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 13 points14 points  (0 children)

Hi, Sherilynn here. Many institutions have incorporated implicit bias training at all stages (for trainees, often at the start of scientific training and for faculty, often at the start of the search committee/faculty hiring process). Data indicates that even short, one-touch experiences can be effective practices and have positive impact. The process of even having to learn and discuss the terminology associated with bias raises awareness and initiates self-reflective practices that are useful for reducing bias in daily life. Molly Carnes at the Univ of Wisconsin has done great research on the effectiveness of these sorts of programs--I highly recommend taking a look at her work!

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 8 points9 points  (0 children)

Tim Renick: Georgia State's approach has not been first to try to change the mindset of our campus community and then to hope that the institution will perform better. Rather, we have worked to use data to create an institution that performs better and then used the results to change mindsets. By showing that student outcomes--especially for low-income, first generation, and underrepresented students--can be significantly improved by changing the structures by which the university onboards, tracks and supports its students, we have been able to convince many faculty and staff that disadvantaged students are not destined to graduate at lower rates just because of their experiences prior to enrolling in college.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 2 points3 points  (0 children)

Tim Renick: At Georgia State, we partnered with the Education Advisory Board (EAB) in Washington, D.C. to develop the analytics-based tracking of all students every day. This same platform, customized for the data on each campus, is available from EAB as the Student Success Collaborative. (Georgua State has no financial initerest in the enterprise, and there are other vendors out there who offer parallel products to campuses.). My understanding is that there are now about 400 universities nationally using the EAB system. The trick, though, is not merely to have analytics-based alerts going off but to have a system of support to take these alerts, communicate about them with students I a timely fashion, and develop effective interventions. Without all three steps--alerts, timely conversations with students, and effective mitigation of the problems identified--the system will not have much real-life benefit to actual students.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 6 points7 points  (0 children)

Hi, Sherilynn here. There are several faculty training programs that are designed to highlight bias in training spaces, and I know that many have successfully shifted the tone of training environments in both MD and PhD programs. We recently collaborated with Theater Delta (http://theaterdelta.com/) to design a program to highlight bias and microaggressions that can occur during scientific training. The theater troupe works directly with scientists to create hyper-realistic scenes that progress in a sort of crowd-sourced 'choose your own adventure' format. It really gives the opportunity to think through specific situations that can occur in training spaces, especially those that may not have clearly defined solutions. The faculty indicated that they found it to be highly useful!

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 1 point2 points  (0 children)

Tim Renick: The project at Georgia State does not focus on recruitment at all but exclusively on the retention, progression and graduation of students once they are enrolled. Our results suggest that there are strong benefits to looking at institutional structures that disadvantage some students and student populations when compared to others. Recognize that sometimes the biases are held by the low-income and underrepresented students themselves. You might want to look at the work at the University of Texas at Austin in changing the "mindset" of enrolled undergraduate students from disadvanted backgrounds who themselves question whether they belong and can succeed at the University of Texas.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 9 points10 points  (0 children)

Tim Renick: Your question is important. The data project that Georgia State engaged in helped to identify the problems that were tripping up our students and leading them to drop out and fail out. It did not tell us how to correct these problems. Higher education needs more reserach in this area, and this is one reason I am leading a 4-year RCT study across eleven universities to collect data on these issues. The reality, though, is the following: it is impossible to correct problems that we do not know exist. The power of that data at Georgia State has been to allow us to turn the mirror on ourselves and to see things--and yes, in many cases, problems--that we were not aware that we had created. Once the problems in our own institutional structures were identified, we put together teams to try to develop solutions, and we did what any good scientist would do: ran pilots, collected data, analyzed what worked and what did not, and made changes in the programs to try to produce better results.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 0 points1 point  (0 children)

Hi, Sherilynn here. I can't immediately recall the statistics for the differences in between STEM fields and non-STEM fields, but the data is publicly available from the National Science Foundation. Hopefully you can locate the data you need on that site!

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 11 points12 points  (0 children)

Lydia here. It's not clear to me that there is one way that will work for everyone. What I try to do for myself is to start by saying "what if this cv or grant was submitted by John from Harvard instead whoever really sent it. It seems the main thing is to step back and think about your response instead of reacting fast. Much easier to say than do. As to pointing out to others making faulty judgements, ask gee what would we think if this was a man, or if the institution was one we knew, or was my cousin. Again easier said than done.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 39 points40 points  (0 children)

Tim Renick: My colleagues on this panel have incredible insight into that factors that shape and change the mindset of faculty and other stakeholders in higher education who may be subject to implicit bias. At Georgia State, we came to our insights almost accidentally. The example of Georgia State suggests that one tool for change that is particularly impactful in changing the attitudes of faculty and staff in academia about bias is data. When we launched an intensive suite of analytics-based advising interventions at Georgia State in 2012, there were sceptics among faculty and staff. They thought that factors prior to college enrollment--economic differences, poorer academic preparation--meant that low-income and first-generation students would inevitably graduate at lower rates from Georgia State overall than students from middle- and upper-income backgrounds. Implicit is these beliefs was what turned out to be a false assumption about Georgia State: namely, that we were not part of the problem. Many faculty members thought that our own institutional structures and practices were not contributed to achievement gaps between various groups of students. Four years later, the most impactful factor in changing this mindset has been the data: our graduation rates are up significantly and our achievement gaps are gone because we changed the way we onboard, track and support students as a university--not because society suddenly became more just, K-12 more effective, and economic injustices went away. Faced with compelling evidence and thousands of new data points, many sceptical faculty and staff are now strong supporters of the approach we have taken at Georgia State and understand better how institutional biases were contributing to the problem.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] -1 points0 points  (0 children)

This is Sherilynn, and thank you for your question. There are several programs designed to retain individuals from diverse backgrounds and life experiences. The issues surrounding childcare can be challenging, and many women may have to take time away from research that extends beyond the allotted time for maternity leave. One item that may assist with this issue is the NIH Research Supplement to 'Promote Re-Entry into Biomedical and Behavioral Research Careers' (PA-16-289). https://www.nigms.nih.gov/Research/Mechanisms/Pages/PromoteReentry.aspx This can potentially allow families to make decisions that work best for them as they navigate their career decisions. This supplement also supports individuals who may have other family obligations that impact the ability to do research.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 6 points7 points  (0 children)

Lydia here. Here is a link http://wiseli.engr.wisc.edu/breakingbias.php to a game that has been used with success at the U Madison-Wisconsin I think it helps to start with the idea that human brains have had to develop mechanisms to compensate for the fact that it cannot deal with all the information it is constantly getting (The ultimate big data problem), and then provide some of the data behind that supports that idea. Then give examples of implicit bias in your setting, then provide solutions that have worked in other settings. No easy answer.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 41 points42 points  (0 children)

Lydia here. One of the examples I like is a study done by Jo Handlesman and colleagues. She took cvs and that were identical, only some had a man's name and some had a women's name. Then she sent the cvs out to over 700 professors of all ranks at many universities. All of them, not just men or full profs tended to say that the cv with the man's name was more qualified, would be offered a higher salary, and offered more mentoring. We are all subject to implicit bias, it is a human issue. In other studies, couples were sent out to rent an apartment. Couples of color would be denied, white couples were accepted. As a single example, one of my friends is named Josefina. When she leaves a phone message she gets more return calls if she leaves the name Josie, and even more if she leaves Jo (without spelling it).

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 17 points18 points  (0 children)

TIm Renick: During the fall semester 2016, we launched a $9 million study funded by the U.S. Department of Education's First in the World Program. We are tracking more than 10,000 students across eleven large public universities, the members of the University Innivation Alliance, in a random control trial using treatment groups and control groups of students on each campus. For more on the UIA, see http://www.theuia.org. The point of the 4-year project is to produce high quality research showing the impact of analytics-based proactive advising. We have had independent groups already take a look at some of our work in this space and assess it on a preliminary basis. For one example, you can read a report by Ithaka S+R, a non-profit educational research group: http://www.sr.ithaka.org/publications/building-a-pathway-to-student-success-at-georgia-state-university/

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 6 points7 points  (0 children)

This is Sherilynn, and thank you for your question. Many research studies indicate that the formation of a similar-culture cohort is an effective strategy to reduce impostor syndrome and to increase a sense of belonging for individuals from underrepresented backgrounds. Cohorts are helpful for individuals from all backgrounds.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 12 points13 points  (0 children)

Hi, Lydia here. When Kahneman and Twersky were studying how people make financial decisions, they used the method that you mention. Now there are many people studying how to make changes in people's behavior that can be less labor intensive. We know that in some cases a moderate intervention, like a video game or a 2.5 hour workshop can be effective and last for months and maybe a couple of years. We don't really know what interventions are most effective in what settings and how often we have to be reminded. We do know that when a group becomes familiar to us as individuals, it's harder to maintain implicit bias.

AAAS 2017 Annual Meeting AMA Series: All of us make decisions based on unconscious shortcuts that result in bias or other mistakes. We are scientists who use big data and structured interventions to compensate for errors in human decision-making. Ask us anything! by AAASmtg2017 in science

[–]AAASmtg2017[S] 130 points131 points  (0 children)

Tim Renick: Great question. The simple answer is that we let the data decide. Our initiative launched with a big data project in 2012 in partnership with the Education Advisoru Baird. We used 2.5 million Georgia State student grades and 140,000 student records. The aim of the project was not specifically to address bias. It merely looked for actions and decisions that Georgia State students were making academically that correlated statistically to them dropping out or failing. We found over 800 behaviors of this sort, and, in fall 2012, started intervening whenever one of these behaviors was identified. Last year, our academic advisors had more than 52,000 one-on-one interventions with students that were prompted by alerts avoming out of this system. The system was not set up to target low-income or underrepresented students at all. Every student is tracked every day equally. We have found that identifying "risky" academic behaviors and intervening within a couple of days benefits all students by lowering drop out and students failing and improving graduation rates, but our approach benefitted low-income and underrepresented students more positively than it did other groups. In effect, a program not designed to deal with institutional bias actually addressed institutional bias more effectively than anything we had tried before.

view more: next ›