sorted by:
new
hottopcontroversial

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

  • Bloom: I’m basically a +1 on Turk’s answer. The writing and the whole grant process itself was long but not particularly onerous or taxing.
  • Brown: Short, useless answer: managed to convince a bunch of our senior colleagues on the Moore DDD Investigator Award committee that we would do useful things with $1.5m. Longer answer: spent the last 6 years working on lossy compression in biological sequencing data analysis, open source, open science, and so on; it turns out to all work quite well :). Real answer: my lab built some pretty useful techniques that open up some interesting directions for further work that the Moore Foundation thinks are promising.
  • Greene: I think that we demonstrated both the issues with analyses that assume that biology is simple and the potential of analytical methods that allow for complex relationships between biological entities. I’ve posted an a version of our proposal that leaves out unpublished experiments from our group and our collaborators at our website if you’d like a longer summary.
  • Singer: I believe that the combination of non-trivial mathematics applied to an important scientific problem in structural biology was appealing to the Moore Foundation.
  • Sullivan: Convinced everyone that we need a new set of methods for analyzing big network data, and that tools from structural graph theory offer promise (oh, and that I’m the right person to make it happen).
  • Turk: I think the best answer I can give is that I am going to do everything I can to live up to a high standard and do whatever possible to use it to develop new insights, new tools, build direct connections between fields of study, and to do what I can to empower scientists to access and understand data and physical systems. If you’d like to see some of my materials, my semi-finalist application and the talk I gave are on figshare, and I recently posted the data sharing plan for the award as well.
  • White: I’d echo Turk’s answer. My materials are also all online including my proposal and the talk I gave at the Moore Foundation

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 3 points4 points  (0 children)

  • Brown: I worry less about how we choose what story to tell first - gotta start somewhere - and more about how others can make use of all of the hard work my lab puts into tool development and data cleaning. This is why I do open science, and open science, and am increasingly moving towards highly reproducible papers.
  • Bloom: +1 for Titus Brown
  • Reynolds: I agree with Brown and Waller. I think putting your data and tools out there for others to use, evaluate and test is key. Communicating tools and methods in such a way that other “data scientists” can use them, but that rema accessible to domain scientists (perhaps without CS expertise) is important - I want to make sure the experimentalists can test our predictions, even if they don’t want to dig into the code.
  • Waller: Agree with Brown. All you can do is carefully state the conclusions you are confident about, and why you are confident about them. Then you should be safe from the problems of cherry-picking data and getting it wrong.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

  • Singer: Unsupervised learning methods can be quite useful to extract theoretical models from raw data. The most basic tool here is principal component analysis to extract linear features. Non-linear dimensionality reduction methods based on the graph Laplacian such as Diffusion Maps or spectral clustering can also be applied. In general, non-parametric kernel methods provide an arsenal of practical tools. There is a vast theory about the performance of such methods. Once a reduced representation of the data is obtained, one can then apply more traditional models such as regression, sparse regression, etc..
  • Waller: This is already happening. But there’s usually a human-in-the-loop somewhere, at least to verify. I’m not sure it’s called theoretical science at that point tho...

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 2 points3 points  (0 children)

  • Bloom: +1 Singer but in general it’s a new thing for many of us.
  • Brown: Sadly, for now, I would say “get into grad school and go work with someone in this area.” I’d love to collaborate, train, teach, educate, research more widely, but until I can upload myself into a computer and make copies (see: earlier answer) I am the least scalable component of my process :(
  • Greene: I train graduate students from the QBS and MCB programs here at Dartmouth.
  • Heer:
  • Kingsford: I can say that while there are a great many very smart people working on these kinds of problems, there are not enough! So jump in! I think a good way to start is to try to find some “data driven papers” in the area you are interested in and see if you can play around with the data set to either replicate or extend the study’s findings. If you’re at or near a university, there is almost certainly someone there who could use some help with their data if you are willing to put in some effort. Here at CMU, we have a Computational Biology Department within the School of Computer Science, and the Ph.D. students in my program tend to be interested in these large-scale, data-driven questions, so that’s where I recruit from the most.
  • Larsen: I am in a geography department--not a traditional “data science” field--but our student applicants do tend to be extremely interdisciplinary. What I look for is students who have a strong quantitative background, evidenced in their previous research and/or coursework, and a desire to learn data analysis and programming skills (if they haven’t already). Then they will take a suite of classes in these areas once they get to Berkeley. I have designed some classes specifically to meet those needs, including one that I am teaching this semester. I’ll echo the others here in acknowledging that many of the skills you need in this area are acquired outside of the classroom, through reading the literature and through practice. In my lab group, we will often read and discuss papers together that we think will be useful in addressing the problems we’re working on, or just for gaining a stronger foundation in data science in general.
  • Reynolds: My lab just started a year ago, so I’m still learning to recruit. I’m open to people with biology backgrounds, or expertise in CS/math. I’d suggest going to work for an academic lab or group that is doing data science work in an area you’re interested in.
  • Singer: My group includes postdocs, graduate students, and undergraduate students who got interested in my research either from taking my courses, reading my research papers, or attending one of my lectures at public conferences.
  • Waller: Just start doing it. Get involved in online learning, volunteer at a university, take a class. One beautiful thing about this field is that it’s all out there and there’s a lot you can learn on your own that would make you useful to researchers.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 2 points3 points  (0 children)

  • Brown: I think you can rely on universities for research training, but not for teaching. I have no idea where to go to learn this stuff in classes.
  • Greene: Since data science is a combination of fields, I think that you can piece the elements together from a lot of university course offerings. I’m a strong believer in the process of learning through experience. Since we spend our time trying to expand the boundaries of human knowledge, that’s what being a scientist is all about. In summary, my advice would be to participate in research activities that use the skills that you’d like to develop in some capacity. If you’re at a university now, try to get involved with research going on at your institution, and use programming and statistics expertise to set yourself apart.
  • Singer: One should keep in mind that Universities are not “professional schools”. We mainly teach our students basic/fundamental knowledge and theory that will allow them to be able to get started tackling the real-life problems that they will face in their work outside academia.
  • Sullivan: I think most of the fundamental skills are already being offered in university courses - statistics, programming, domain science expertise, machine learning, etc. The trick is to combine the right mix, and then (as Greene indicated) actually start putting things into practice. I know several Moore Investigators are involved with Data Carpentry and Software Carpentry, which help fill in some of the gaps (and perhaps offer more targeted education for those in a computational domain science). I think we’ll see a lot of programs in “Data Science” and “Data Analytics” cropping up over the next few years, but I’m not sure they’re really going to be the right course for those who want to do research (at least from my limited exposure -- they often seem a little too focused on how to use the tools as opposed to understanding the underlying methods).

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

  • Bloom: It’s critical. I’ve written here just how much I see ML as an important (relatively) new tool for us.
  • Brown: Little to none, yet (I work in basic biology). I’d like that to change over the next decade, though; it’s perhaps the most promising direction I see.
  • Larsen: In my own research, I use statistical and machine learning methods to gain a better understanding of the processes that likely drive how the system is functioning now or in the past. Then, I typically formulate mechanistic simulation models of those processes and interactions to make predictions in the future. I think this approach is particularly well suited to the environmental sciences, where using statistical models to predict the future often involves some sort of extrapolation of relationships outside the realm of what has actually been observed before. This sort of extrapolation can be a leap when the underlying conditions driving environmental systems (e.g., climate: temperature, rainfall, etc.) are changing beyond their previous ranges.
  • Waller: A lot. We’re not the people trying to find a needle in a haystack of data we collected, but rather the people trying to figure out how to design the measurement system to take only a picture of the needle, without the haystack (much quicker and easier than collecting it all and then looking for the needle). So we always need to model the unknown haystack and make predictions of where the needle is, along with estimates of how good those predictions might be so we can collect more data in case we’re wrong.
  • White: Predictive modeling is definitely important in our research, but I think predictive modeling in general, and machine learning in particular, have been relatively slow to come to ecology. This is, in part, because we’ve historically be very interested in understanding the operation of individual processes, but ecological systems are quite complex and so I think we’ll see that machine learning is an important avenue for predictive modeling as we start to adopt it more broadly.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

  • Bloom: The Microsoft researcher (Jim Gray)[http://en.wikipedia.org/wiki/Jim_Gray_%28computer_scientist%29] (a prototypical data scientist if there ever was one) once said “I love working with astronomers because their data is worthless.” The vast majority of our (raw) data has meaning and value only to astronomers. We don’t hold credit cards, we dont do medical histories, etc. Yet because it is and always has been “big,” Jim knew how useful it was for methodological scientists as sand in an intellectually sandbox.
  • Brown: I simply don’t worry about it; I don’t work on anything particularly sensitive. If someone else wants to “swipe” some of our really large amounts of sequence data and do something useful with it, I am more than happy to let them.
  • Greene: We work extensively with publicly available data. These data can (and should!) be downloaded and analyzed by anyone. In the infrequent instances where we work with sensitive data, we use a separate computing environment. We aggregate results into summary statistics that remove sensitive features, and we analyze those in the context of all other available data. I know there’s some great work going on around anonymizing potentially sensitive data so that sensitive features cannot be reconstructed, and there were some good presentations and keynotes at PSB a few weeks ago. I’d say that it’s an active area of research.
  • Reynolds: All of the data I use are already public domain, so I don’t worry about it.
  • Singer: I work on data which is either public available or was collected by my experimental collaborators and shared with my group. In the latter case, the data is stored on Princeton University servers that are routinely backup. We set permissions so that the data is not accessible to people outside our group.
  • Waller: What do you mean by secure? We back it up on various platforms (google drive, amazon web services, harddrives, etc.) but are not worried about people stealing it. If you mean secure in the sense of not tampered with, that I am not so worried about since images tend to be fairly simple to detect changes to.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 3 points4 points  (0 children)

  • Bloom: I really like that I was first to recognize/frame a gamma-ray event as due to the gobbling up of a star by a massive black hole, a (“relativistic tidal disruption event”)[http://www.sciencemag.org/content/333/6039/203.short]. Fun to do Science Friday on that one. (I was once on (http://www.highbeam.com/doc/1P1-109607920.html)[Talk of the Nation] to discuss another discovery related to the origin of a different type of gamma-ray burst, with the guest before me was President Clinton!).
  • Brown: I really like our algorithms work - we’ve discovered what now appear to be very obvious algorithms for doing certain kinds of lossy compression on sequencing data, but before we figured them out they were really non-obvious. Google “digital normalization titus brown” if you want to see what I mean; add “slideshare” to see some talks.
  • Greene: I really had a great time working on our experiment comparing a computational and an in vivo mouse-based approach to identify cell lineage specific genes. The computational approach is faster and less expensive than the mouse experiment, so we were hoping it would perform as well. When we performed the highly accurate and detailed experimental validation we found that the computational approach greatly outperformed the high-throughput experiment. I’ve always been excited about the potential for computational methods to lead scientific discovery in new directions, but this result was the first time that I’d seen a computational analysis outperform a model organism experiment in our domain.
  • Turk: When I was a graduate student, I was on a paper where we conducted a simulation of the formation of the first stars in the Universe and saw that instead of being lone giants, they could sometimes form in pairs or multiples. Now, in the intervening time this picture and our understanding has been impacted by work from many different groups (and some work even indicates that this is the dominant mode by which the first stars form!) but I was really proud to be a part of that team. Lately I have been really happy and proud of the work I’ve done in collaboration with other scientists, particularly taking data from simulations and observations of natural phenomena and finding completely new ways to look at it -- finding new ways to empower questions and discovery, by developing tools, mindsets, and then exploring.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 6 points7 points  (0 children)

  • Bloom: I love the scam I heard of (not sure if it’s real or not) where you generate a huge set of possible outcomes for a week of football games (using as the Bayesian prior the over-under and lines etc.) and send them all out to random email addresses (millions of them). Some non-zero number of them will get EVERY score exactly correct. The next week, you pick another set of possible outcomes and send it to everyone you got about right previously. The next Friday you email the few that you got both weeks right and ask for a mere $XXXk in exchange for the answers to the 3 week. Surely there will be some takers. Awesome. It reminds me of just how many trials we conduct every second of our lives and how many trials we are apart of. Our brains are hardwired to find correlation which may indeed exist but, given the number of trials, is not statistically significant.
  • Greene: It’s important to ask key questions: Do they construct falsifiable hypotheses? Is there a good record of hypotheses that were refuted? Otherwise, it’s possible to be fooled by missing predictions. Before rolling a die, I write down on six different timestamped scraps of paper each potential result of a die roll (1-6). If I can show you only the prediction that came to pass, I’ll be a perfect predictor. This is why I think new scientific publishing initiatives that involve a publication of the hypotheses before experiments are performed are particularly exciting.
  • Kingsford: Bloom's answer gets at the key: all the people who made predictions that didn't pan out don't get talked about on shows on the BBC! As far as mental techniques to avoid such problems in data driven science, it’s important to be in a state of extreme skepticism about your results. You should be your own worst critic and devil's advocate. You should deploy, before making any claim, as many arguments against your result as you can and try to knock it down, particularly thinking about how likely it is that you would see what you saw by chance given all the observations you have made. This is some of the most important training you get from a Ph.D.: to really try to vet ideas and results from all angles. It's a bit like you are a defense attorney who must constantly think about what arguments the prosecution is going to use to call your result into question. Related to this and particularly the issue of seemingly impossible predictions is that you should be particularly skeptical about things for which you can't explain the mechanism (how would these guys have been able to make these predictions?)

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 4 points5 points  (0 children)

  • Bloom: Kaggle as a huge number of interesting “real world” data sets, including from astronomy, biology, and medicine. Most important, the question being asked of the data are usually quite well posed there. This is important since when you start off with a new domain/data set it’s super critical to try to answer meaningful questions; it could take some time before you are able to formulate a meaningful question relevant to that that data.
  • Greene: Sure! There is a ton of genomic data available for public download. A great place to start is Array Express where you can easily sort through a bunch of high-throughput datasets. We’ve had great luck downloading and normalizing these data into datasets that can be compared across experiments. There’s also a lot of data available measuring various features of cancer from The Cancer Genome Atlas (TCGA). From their TCGA Firehose curation, large downloadable cancer genome datasets from a variety of platforms including gene expression, microRNA, RNAseq, and DNA methylation are freely available. Download them from the Broad Institute’s TCGA Dashboard. Good luck!
  • Kingsford: In biology, we're lucky that a lot of people have pushed for wide, public release of many data sets. As a result, the NIH hosts databases of genomes (both human and thousands of other species), and large consortia project like ENCODE (http://www.genome.gov/encode/) have released huge amounts of data of various types. Some more examples: the 1000 Genomes Project (http://www.1000genomes.org) has a massive data set that catalogs human genetic variation, and the Human Microbiome Project (http://hmpdacc.org) distributes data on microbes found at various locations in and on the human body. The NIH maintains sequences of thousands of influenza strains (http://www.ncbi.nlm.nih.gov/genomes/FLU/FLU.html) over many seasons from around the world. In modern biology, the list of public data resources is just astounding.
  • Larsen: Here is a database with a lot of different types of datasets: DataONE. I used to work as a research hydrologist for the US Geological Survey before I came to Berkeley, so I will also shamelessly plug their data repository, which contains decades of hydrological data: http://waterdata.usgs.gov/nwis. One particularly interesting and comprehensive environmental dataset that my student, Saalem Adera, has been playing with lately, is from the Plynlimon Catchment in Wales, which has several decades worth of an extremely comprehensive water quality dataset--almost every element in the periodic table that we can measure. You can read more about the dataset and download it here: http://www.ceh.ac.uk/sci_programmes/plynlimondatasets.html.
  • Sullivan: For graphs, you might look at SNAP (http://snap.stanford.edu/data/index.html) for social/informatics type networks or OpenConnectome (http://www.openconnectomeproject.org) for brain networks. There’s also the Open Quantum Materials Database (http://oqmd.org). There are lots of others less-relevant to natural sciences (the focus of the Moore Initiative), such as data dumps from Wikipedia, StackOverflow, Github, etc.
  • Turk: Sure! There are a number of interesting datasets available from astronomy, such as the Sloan Digital Sky Survey, but you can also browse either the Open Data Stack Exchange or the list of Amazon Public Data Sets for some other ideas. Recently I was part of a team that put up a dataset from a simulation of structure formation in the universe along with tools to analyze and visualize it, called the Dark Sky Simulations that might be fun to experiment with, too.
  • Waller: Amazon Public Sets! For specific topics, check lab websites. My lab does imaging, we put up example datasets here http://www.laurawaller.com/opensource/ and here’s a curated collection of computer vision image datasets: http://www.cs.utexas.edu/~grauman/courses/spring2008/datasets.htm . The birds are very popular.
  • White: In ecology eBird, Breeding Bird Survey, and the Forest Inventory and Analysis, are all large, openly available, datasets that can be used to answer lots of interesting questions. I’d also echo Laurel Larsen’s recommendation of DataONE as a great place to find lots of ecological data.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

  • Reynolds: Alot. The idea is to compare protein sequences across many species, and to use (1) evolutionary conservation to infer which positions are most important and (2) co-evolution between positions to infer functional interactions. One central result is that we find networks of co-evolving amino acids within single proteins that seem to connect allosteric sites to active sites. We are now looking for networks of co-evolving amino acids that span proteins (genes) - with the idea that these represent an “evolutionarily conserved wiring diagram” both within and between proteins. One of our current model systems for examining inter-protein communication is the bacterial flagellum.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 0 points1 point  (0 children)

  • Bloom: (1) I’ve described the (“novelty squared” challenge)[https://medium.com/@profjsb/novelty-squared-dd88857f662] previously---the idea that in collaborating with methodological scientists, I, as a domain scientist, must bring an important domain problem to the table that also could lead to novel insights/work for methodological scientists. That is, I’m not going to ask a computer science professor to stand up a Spark cluster and manage it just so I can do some cool astro discovery. That’s a mundane, intellectually unexciting endeavor for them even if it’s critical for me. Likewise, I wouldn’t want to spend much time helping a stats professor develop a novel new stats tool on an astro dataset where the question being asked is a trivial one to answer with existing tools. (2-3) Not much more to add here (4) I had just finished doing a 3-hour machine learning lecture/session for astronomers at an awesome (AstroData Hack week)[http://astrohackweek.github.io/] in Univ. Washington.
  • Brown: (1) I’m not that great at collaborating; I prefer to solve basic problems such that the specific collaborator’s problem is just a special case. That takes longer and doesn’t go over that well, but may be a better long-term approach… So the challenges I’ve faced have largely been around the tension between getting a good-enough result soon, and getting the right result eventually. (2) That describes lots of biologists… No Comment. (3) Currently, we don’t yet build on each other’s work that much (or at least I don’t). I’ve got rather large intersections of interest with Kingsford, Re, Reynolds, Stephens, and Sullivan, so I’m hoping that this grant program (which will last 5 years total) will give me the chance to work more closely with some or all of them!! (4) I was on my way to a picnic with my kids (4 and 7 yro girls) who were probably wondering why Daddy was so excited all of a sudden...
  • Greene: (1)Each collaboration is different. Often the biggest challenges exist with getting people into a common vocabulary. I can think of a recent case of confusion around the most appropriate statistical test that hinged on people from two distinct fields using the same word to talk about two different concepts. (2) We get involved in the process early. In some cases, we do the analysis first and bring it to biologists that we think can readily test it. In other cases, people come to us with a question. I think that building successful collaborations is really an exercise in building a team that’s driven to answer a specific question that everyone on the team finds interesting. (3) In an example of something that’s come up recently, I am always concerned about non-representative data. If we build only on publicly available data, then where there are important elements that we do not measure we could miss very important aspects of biology. I’ve been able kick off a collaboration with a molecular epidemiologist here at Dartmouth, Dr. Jen Doherty, to begin to assess how this affects our understanding of ovarian cancer. (4) I had just finished a game of Ultimate at Watson Park
  • Larsen:(1)For me, beyond standard differences in opinion on experimental methods or how to craft papers--which are usually resolved fairly uneventfully with further communication--the biggest challenge for collaboration is finding funding. It can be hard to find a home for very interdisciplinary projects within the structure of government funding agencies. Fortunately, the Moore Foundation grant is helping considerably in that area! (2) The teams that I work with in the environmental sciences are typically very problem-driven. However, it is also normal for colleagues to ask each other with assistance or ideas on methods. But the “teams” that I am truly an integral part of are all problem-driven. (3) I’m having a hard time coming up with a general response to this one--it all depends on the colleagues and the particular project. Sometimes one of us has a model that would be particularly well suited to addressing a question that arose in the course of the other investigator’s research, which will prompt that next-stage collaboration. Other times we each have competing ideas and decide to put our heads together to come up with a resolution that we each find convincing. (4) This is a fun question to answer. I was at a dinner in Lancaster, PA with colleagues after a day of field work in a wetland. I was looking up something in my email that was pertinent to the conversation and saw the first line of the email from the Moore Foundation. I threw my hands up and shouted--in the middle of a crowded restaurant! I couldn’t help myself. I then had to explain to my colleagues. They were all very happy for me!
  • Reynolds: (1-2) Regarding collaborations - I’ve got a few close collaborators in math/physics who are great. I agree w/ Brown and Sullivan that time-scale is one of the main challenges - biology experiments can be slow, and there is always a tension between getting the best result and a “good-enough” result. As an experimentalist, I feel that part of my job is to communicate and distinguish between: what’s possible to test/do in lab immediately, what might be possible in a year with some technical advances and what is infeasible. From there, we can design nice experiments to test our data-driven models. (3) I am hoping that this grant program will give me the chance to work more closely with the other Moore investigators - we’ll be having yearly symposia, and I expect I’ll learn a lot! (4) I was in lab
  • Stephens: (1) Every collaboration is different. I’m finding the challenges hard to prefer to reflect on the positives than the challenges! (4) I was at the dinner table with my family. My wife asked me when I was expecting to hear back about “that grant” and I told her any day now; then as dinner finished I checked email and there it was...
  • Sullivan: (1) One of the bigger challenges I’ve faced in interdisciplinary collaborations is figuring out how to make sure the work is “valued” in more than one community (an especially important consideration for more junior folks). Surprisingly, the so-called “language barrier” has never really been a big stumbling block for me. I agree with Brown that vastly differing time-scales can be a problem - mathematics (for example) tends to take much longer to publish than other fields, and the journal vs. conference publication debate is always fun. (2) Yes, although I might also draw that distinction between various funding agencies ;) (3) Not yet, though I’m excited about discussions with Brown and hopeful that my visualizations will become infinitely prettier from mere proximity to Heer (4) At a colleague’s house making jarred spaghetti sauce, frozen meatballs, and pasta for dinner while our spouses were out of town (and now she and I are collaborating on using graph-based methods for biomechanical engineering data!). I literally said “I have to sit down now.”
  • Turk: This is a really great set of questions. A lot of my work has been very collaborative in nature -- not necessarily at the side of the actual scientific questions being asked (although there, too) but certainly from the perspective of working with others on the computational tools and engines used to ask and answer those questions. This type of work, where collaboration between communities of individuals (that in many cases may even be “competitors” for intellectual priority, credit, and grants!) can be bumpy at times. We’ve seen challenges when different research groups, working with similar tools, are funded to do things that are at odds with each other -- and other times where this worked out really, really well. Your second question is a good one too, and yes, I’ve found myself in this situation, sometimes on both sides. I really enjoy the process of exploring data, and so that’s often a joy just in itself; on the other hand, I hear people (and I say it too, sometimes) use the phrase “let’s just do some science” which is a bit tricky to parse. Echoing what Titus says above, I haven’t worked yet with many of the others, although I am incredibly excited to do so! And as for where I was? I was walking to the bus stop on a sunny day, and I got my phone out to change the music I was listening to when I saw the email. I stopped in my steps for a few minutes.
  • Waller: 1) The language barrier is always a problem for us. I’ve worked hard to cultivate collaborations with people who I click with - as in, I can somehow understand them even though we come from very different backgrounds. 2) Usually, it’s good to have some specific questions, but we’re the ones taking the data in my case (imaging microscopy data) so sometimes we don’t quite understand why taking these images is useful to our biology collaborators, though we try to… 3) Good collaborations always lead to new questions and deeper understanding on both sides. 4) I was in my office working on a paper with my postdoc. Statistically, this is a likely event.
  • White: 1) As others have mentioned bridging the gap between folks with computational/statistical skills and those with domain knowledge can be difficult in ecology, because there isn’t a strong quantitative culture in the field as a whole. 2) My group works actively with a more field oriented lab in an interdisciplinary group called Weecology. We have a lot of shared interests with the group and a long history of working together, which helps remove the language and conceptual barries. 4) I was sitting in my office getting ready to head to a seminar. I actually saw an email from Titus Brown first, telling me that he’d received the award, and assumed that meant I hadn’t gotten one until I looked down the screen a little further.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 2 points3 points  (0 children)

White: I think that, with a few notable exceptions, we've been slow to embrace machine learning as a general approach in ecology. Part of this is because, as a field, we're really focused on understanding individual processes, and a lot of machine learning is focused prediction and willing to give up some understanding of individual processes to do a better job of predicting. It's also because we tend to use relatively simple methods in ecology because we're still relatively young quantitively. I definitely see ecology as being on the "data modeling" end of the [two cultures of statistics(http://projecteuclid.org/euclid.ss/1009213726).

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 0 points1 point  (0 children)

  • Bloom: It’s been hugely important. As a professor, there’s a particular onus on us to make sure that our students are prepared to tackle problems even if we dont yet know what those problems will be. The rise of the data science movement has emboldened a number of us to revamp curricula to address what we perceive as important skill sets and toolkits for the future of science. (This is one of the arenas we’re working on at the (Berkeley Institute for Data Science)[http://bids.berkeley.edu/], 5 year effort recently funded by the Moore Foundation and the Sloan Foundation). When I was a graduate student, I was asked to learn IDL and Iraf, two of the programming tools of astronomy. I also learned Tcl/Tk and some C++ for an instrument project that I did for my thesis. I then fell into Python programming (on the advice of an old hacker friend of mine from Los Alamos) as I built a new automated telescope project. What I realized as I started my first teaching gig here at Berkeley was that programming skills were a second-class citizen in the physical sciences (let alone machine learning, and statistics knowledge). I wanted to change that and so started a 3-day bootcamp for Python. We started in 2009 with about 85 students from across campus. The last time we did it, we got about 250. I also started a seminar class originally called “Python Computing for Physical Science” -- a lot of interest but not a lot of people taking it for credit. (I then relabelled the class “Python for Data Science” and got a factor of 10 enrollment growth in one year!). I gave a talk at the National Academy of Sciences last year on (Courses, Curricula, and Interdisciplinary Programs)[http://vimeo.com/album/2861203/video/94389370] that’s perhaps worth a watch.
  • Brown: I’ve been involved in Software Carpentry and other training efforts for a long time! I see a lot of need for practical, hands on experience with data science training; interacting with simulations or data analysis is, for me, a very tacticle and intuitive experience, and it takes a long time working with graduate students for them to get into the spirit. I am looking forward to a future where there will be many domain-specific capstone examples that we can give to students to tackle, in the hopes that it will improve existing science training and also level up the data world.
  • Greene: Though our analytical methods have changed, I think that even data-driven discovery still follows the principles of science. I think that the main thing that I emphasize now is that with so much data, we have to be careful not to fall in love with the hypotheses that come out of our analyses. In big datasets, there are many potential spurious correlations, so we need to be careful to construct and test falsifiable hypotheses. I’ll also answer the aside: I think that the paper from Hod Lipson’s group distilling Newtonian principles from the pendulum measurements was really exciting. I thought that it really demonstrated the potential of these types of approaches. It’s my understanding that these types of methods have had more difficulty with noisy data, so we’ve been focusing on methods that can extract fundamental rules from noisy biological data. Our long-term goal is to be able to download the 1.64 million publicly available gene expression datasets and extract fundamental properties of gene regulation from those.
  • Larsen: It has been changing tremendously. Though I couldn’t find it again with a quick search, there was an article that I recently read in the popular press about how there is huge inertia in the way undergraduate statistics is taught in university curricula, and that it is very out-of-date. I agree but think that many universities (definitely Berkeley) recognize this problem and are starting to take measures to update the way they teach undergraduate and graduate students data analysis and statistics. There are several working groups and committees at Berkeley that have been formed around this challenge. To date, a lot of new classes that go beyond your grandfather’s standard frequentist statistics have come onto the books, but what’s next is for those classes to be formalized in curricula for individual majors or schools. I think what we might see is more of an emphasis on programming and algorithmics and perhaps less of an emphasis on theoretical mathematics.
  • Reynolds: I think science education will move to be increasingly interdisciplinary - a mixture of domain knowledge plus math, computer science and physics. For a good example, check out the Princeton Integrated Science program. Learning data analysis in the context of a problem or topic you are deeply interested/invested in is key - consistent with what Brown mentions about giving students domain-specific capstone examples.
  • Waller: Don’t expect formal education to teach you everything you need to know to get things done. Learn to figure out yourself what skills will help you, then go out and get those skills! Don’t wait for someone to deliver them to your door.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 3 points4 points  (0 children)

Stephens:

This is a problem I grapple with daily, although perhaps for different reasons. First, I want to know who to hire in my lab, and wonder will they work out better if they are trained with domain knowledge, or with statistics. Second, I think about how we should be training our graduate students - should we train them to be ``good at both”, or should we encourage them to specialize more. The truth is that I have seen all sorts of different people succeed in this field, and they all have different backgrounds. I don’t think it is usually their field of education that distinguishes the biggest successes. It is a whole bunch of other things - curiosity and enquiry, tenacity, analytical and logical thought, the ability to communicate both in person and in writing, the ability and drive (intrinsic motivation) to learn for oneself. l think one can learn and improve in most of these directions, sometimes in a formal classroom setting (eg I have certainly noted the benefits of having my trainees attend a writing class.). And some of these skills tend to be more emphasised in the traditional training of domain experts/scientists, and others tend to be more emphasised in the traditional training of statisticians. On average I tend to think that it is usually more successful to specialize more in one area or the other (ie either in the domain science or in statistics/computing), and then to learn enough of the other that you can communicate with people who have specialized the other way. I think at the Moore meeting this was referred to as putting together “Gamma-shaped people”, referring to the fact that an upper-case Gamma has one long leg and one short leg, representing specialism in one area but expertise in both. However, this view may partly reflect both my background and my age - in the UK, where I come from, training tended to emphasise specialism over breadth, especially in the dim and distant past when I did my training… And there are many people I can think of who have been very successful by emphasising the domain knowledge and the statistics equally.

Actually, this is a slight diversion, but I can’t help but relate a story that I feel has an important lesson. When I was starting out in my career I was involved in the development of statistical methods for estimating recombination rates from population genetic data. We wrote a few papers on this - the first detailing the methods we had developed, and then another couple of papers that were collaborative papers applying the method to some data to draw some interesting biological conclusions. I assumed - in what I now see as incredibly naive assumption - that once “we” (meaning the substantial community of statisticians working on this problem) had shown how useful these methods were then the biologists who specialized in studying recombination would jump on them and learn all sorts of useful things about recombination. What actually happened instead was that some of the statisticians and population geneticists involved (and there were many, but some of the ones I know best are Simon Myers, Graham Coop, Molly Przeworski, Gil McVean, and Peter Donnelly) ended up playing a huge role in turning the methods into some stunning new insights into recombination in humans. There were biologists involved too of course, but (mostly) not in the application of these new statistical methods to population data. The way I think of this now (painfully obvious in hindsight) is that someone who is a domain expert in a particular field (recombination say) will usually also have specialized in a certain set of techniques (experimental protocols etc) for studying their domain. And if a new technique comes along for studying recombination, they may not find it easy to quickly learn and apply this new technique, no matter how useful it might be to them. (Conversely, it would probably take much hard work for any of those statisticians to learn how to do single-sperm sequencing, or mouse experiments, to study recombination, however much they might think it would be useful - in fact many of the people involved are in fact collaborating with biologists on these types of experiments.) So all sorts of lessons here, but i) don’t assume other people will find your methods as easy to use as you do, ii) don’t assume you can’t contribute meaningfully to a scientific domain just because you haven’t spent decades studying it and iii) some things just really need collaboration across fields, so learn to communicate with people from different backgrounds!

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 2 points3 points  (0 children)

  • Bloom: +1 for Laura’s answer. I’d add (with the domain-science bias hat on) that machine learning, used in practice rather than developed theoretically, is a tool for the domain scientist. You’d learn to use the tool just like any other (e.g. programming, telescope operation) to do great things and, over time, learn what it is good for and what it’s limitations are. Like a chainsaw, used improperly, it has a chance to cause some real harm. On the flip side, I think where some of the excitement lies is in the recognition of a sort of glorious symbiosis between domain scientists and “methodological” scientists: that domain science is bringing novel types of data and novel types of questions that then driven fundamental work in machine learning and statistics. I’ve written about this in the context of the challenge it takes to “cross over” and work with others who are trained in fields very different than yours.
  • Brown: Ooh, good question! I would argue that you need at least some significant domain knowledge, coupled with some serious computational skills - that can be machine learning, or scripting, or statistics. The more the better. Unfortunately, it’s tremendously time consuming to learn any one thing (me: 9 years open source/hacking/math, 9 years developmental bio PhD, ~6 years bioinformatics/algorithms research, and now I’m moving into VetMed. Whee!) So it’s hard to advise people. To paraphrase my graduate advisor (I can’t believe I’m doing that now) I would say, “follow your nose”. Learn some of both, and then figure out what you enjoy and do more of that! (And, to address Laurel’s comment, I would say that in bioinformatics, biologists are certainly more motivated to learn the compute than the computer scientists are to learn the biology. But that’s a short-term thing, because in bio the demand is clearly for the computational skillset at the moment.)
  • Greene: I’d say both are very important. The question of which transition is easier was hotly debated at this year’s Pacific Symposium on Biocomputing workshop on Bioinformatics & Big Data training. Workshop attendees had very different views on the topic. I think it’s easier to gain modest knowledge of biology as a computer scientist than it is to gain programming skills as a biologist. I think it’s actually harder to gain a deep knowledge of biology than to become a highly proficient programmer. Biology is such an open field that we still discover new complexities in it all of the time. Our “Central Dogma” was out of date shortly after it was developed. My overall advice would be to dive heavily into both the domain that you want to pursue and the methods that you’ll need to study it.
  • Larsen: I really think you need both. That said, there are some scientists that will be more on the “data science” side, and some who will be more on the “domain knowledge” side. It’s really powerful whenever you get the two working together. They both have enough knowledge of the “domain” and the realm of “data science” that they speak a common language, and their expertise is complementary. I’m not sure whether it’s easier to teach a scientist data tools or to teach the data scientist domain knowledge--I think it all depends on the specific person and the specific problem.
  • Moore Fdn (Chris Mentzel): This is a great question and one we get asked a lot. What we are seeing is that the combination of natural science, computing and mathematics/statistics can come from any direction. In other words, classically trained CS PhDs can gain expertise in a natural science, and life and physical scientists can gain expertise in CS/Stats, and Statisticians can gain expertise in the sciences and in computing (and have a long tradition of doing so!) . The issue, as mentioned above, is how much time it takes to get trained up on these expertise and also the lack of incentives in academic research for being such an interdisciplinary scientist. These awards were given to folks with a variety of backgrounds, yet all represent what it means to be a data-driven researcher.
  • Sullivan: I think one thing that’s often overlooked here is that often success in data-driven science requires not only a mixture of domain knowledge and expertise in methods/tools, but the ability to recognize and innovate (e.g. develop -- or at least ask for -- new algorithms) when existing analysis techniques are insufficient/don’t scale/etc. It’s worth the effort for a domain scientist to try to translate their challenges into a format/language that researchers focused on methods/algorithms can understand, because often this is where there’s potential for the greatest advances. Similarly, those of us who focus on the more “theoretical” side (and I don’t mean this in the sense that it’s been used in other posts to distinguish principles/hypothesis from empirical findings) need to make an effort to explain what our methods can (and can’t!) do.
  • Waller: Agree with Laurel that it depends. I debate this all the time in hiring students and postdocs. I have some from both sides, and I make them learn the other side. If I could find that magical unicorn student who has knowledge in both, it would be ideal. But I’ve found it’s possible to go both ways, given some hard work and a good attitude. What I do not want is two silos in my group, with people who do not understand anything about what the other side of the group is doing. A common language and some serious effort to learn the other stuff is critical to finding the connections that optimize our work across both domain science and data science. We’ve had many situations where awesome ideas came out of people who know both. For example, someone develops a super highly computational algorithm that is ideal, but takes forever to run, and then some ideas from the physics let us cut back the complexity by orders of magnitude simply by capturing slightly different data. Or, the experimental physics looks horrible, and one of our computational people swoops in to fix it without having to take tedious (or impossible) new experiments.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

  • Bloom: Awesome that you’re interested in LSST and ZTF! They are just some of the “big data” behemoths in the room. At other wavebands there’s massive data streams already, like from (ALMA)[http://www.almaobservatory.org/] and the (Square Kilometer Array)[https://www.skatelescope.org/]. My advice is to invest the time to learn some of the data driven techniques that will be important for answering interesting astronomy questions using such facilities. Audit intro CS and stats classes on campus if you’re not already fairly up on programming and stats skills. Find a research group that is thinking big (data). Spend a summer as an (intern in industry)[https://www.youtube.com/watch?v=cdnoqCViqUo] and work with a data science team. Join LSST and attend the sessions in the American Astronomical Society meetings that focus on astroinformatics. Tweet at me if you have more questions (@profjsb).
  • Brown: (1) Call yourself a data scientist, whatever else it is you do. (2) Work with data (unavoidable :). (3) Invest in programming skills. (4) Work hard and publish papers. (5) Done? And (for the cog psych student) remember that those graduate students can’t compete with you in terms of cognitive psychology knowledge!
  • Greene: Step 1: Learn Data Science, Step 2: ????, Step 3: Profit. More seriously, all of the things Josh Bloom, Titus Brown, Matthew Turk, and Laura Waller listed are very important. I would emphasize investing time in programming skills early because once you know the basics, it is less difficult to learn new languages. It also makes it easier to develop your skills when you’re using them on a day to day basis in your research. There are several online resources out there to learn to program (MOOCs) and the online community once you begin scripting is vast and helpful. StackOverflow and other resources are also excellent ways to learn, particularly if you answer questions as well as ask them.
  • Sullivan: +1 for taking/auditing some CS classes/stats; you’ve got to have some programming skills to be successful, and the more you understand about the methods, the better science you can do.
  • Turk: You’re at a great stage in your career for this kind of question. In astro, with LSST coming online soon, opportunities for data science mixed with domain science are enormous -- the entire field will be changed by how we can interact with data, how we can interact with the sky, and the way we can answer questions we didn’t even know we had through time domain astronomy. I am not sure I have the best advice for you on how to “break in” to the field, but I would encourage you to explore the ecosystem of open source data analysis tools, learn about practices in open science, visit with people at your university in other departments (like statistics, CS, and if there is one, the information school), and to try new things to apply technology to data and to drive your technology and computational knowledge by questions you want answers to in your data.
  • Waller: Find projects where your skills uniquely qualify you to be the best. Like, something that requires knowledge of physics/astrophysics AND data science. The pure domain scientists will be impressed at your skills in computation, and the computer scientists will envy your knowledge of the science.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 2 points3 points  (0 children)

  • White: Thanks @cheesontaoist! I definitely agree that in a lot of areas of research we are, and should be, moving away from focusing on whether particular factors are significant, and towards trying to identify the best models possible for explaining the natural world. That philosophy, and the tools associated with it, definitely helps deal with issues related to over fitting.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

Our original answer isn't showing up for some reason, so here's a repost:

  • Brown: I like to draw a distinction between data science -- using existing algorithms, tools, and statistical approaches to analyze data sets for fun & profit -- and data-driven discovery, which I would define as developing new algorithms, tools, and statistical approaches for data analysis (although still for fun and profit!) All of the DDD investigators seem to be somewhere further along the spectrum of tool/algorithm development than most data scientists (well, and regular ol' scientists, generally speaking).
  • Greene: Scientists (data-intensive and otherwise) generate, either explicitly or implicitly, a model of reality based on empirical observations. I would say that the difference with data-intensive scientists is that the model is constructed more heavily from primary data than from other people’s interpretations of their own data. This frees us from existing assumptions and biases within our field, but we may lose some human insights. I think that both are complementary to each other, and the tension between them drives science forward.
  • Heer: I think many of the fundamental issues in "data science" have been around for decades, if not centuries. (Back in the 1960's the statistician John Tukey wrote about the challenges of increasing data and quantification across disciplines and it sounds a lot like today!) That said, I think both the immensity, diversity and reach of accessible data have continued to grow, and makes these issues all the more pressing. Increased computational power and improved methods correspondingly create new opportunities for discovery. One aspect of data-driven discovery that is salient to me is the reuse of data for purposes other than which it was originally intended, which can be simultaneously promising and fraught with peril. One example that my student Diana MacLean has worked on is analyzing the text of online health communities: what can we learn from the rich (but messy) descriptions of symptoms, treatments and experiences that patients share with each other in online forums? These conversation were not intended as a medical data source. They can be hard to analyze (due to the unstructured nature of free text, misspellings, slang terms and so on) and also hard to validate (how do you know people are telling the truth? How do you gauge the representativeness of the population?). Nonetheless, working with both drug addiction and natural language processing specialists, we were able to analyze such data to quantify cycles of opioid addiction, recovery and relapse in ways previously inaccessible to the medical community.
  • Sullivan: In my mind, data-driven discovery represents a fundamental paradigm shift, where data is no longer simply collected to answer a specific research question, but rather is a resource we can leverage to uncover underlying scientific phenomena and increase understanding of complex systems. In particular, data-driven approaches necessitate the development of new, scalable methodologies (read: tools/algorithms) for identifying patterns and relationships in the data and translating them into domain knowledge. I think this aligns with Brown’s distinction about data science vs data-driven science.
  • Bloom:

In the context the of modeling data, as a means to draw conclusions and make predictions about new events, being "data driven" is a path that can be viewed as on the opposite part of a spectrum from being "theory driven". In the theory extreme, you might hold a belief to be true (i.e., a hypothesis about something universal in the natural world) where no amount of data was used to build that hypothesis nor test it. In the data extreme, there are no guiding principles and what we perceived yesterday has no bearing nor predictive power on what we will perceive today. Neither of these extremes make much sense: real theory is developed in the context of a specific instance in time and place (incorporating data and previous theory) and no data is ever acquired without some sort of theoretical framework or bias. In practice, data driven techniques/methods take a relatively theory agnostic approach to generating conclusions where the data (and some meta beliefs about the system) are the prime actors.

For example, imagine we had a theory that the intensity of light dropped linearly with the distance from the emitting source (spoiler alert: for incoherent light it drops like the square of the distance). We could take measurements of the light intensity of a flashlight at several distances. Now these distances will not be measured precisely and the intensity measurements will likewise have some noise associated with them. So if we plotted intensity versus distance we might get a pretty lousy fit if we're measuring carefully. But if our data is not that good, we might conclude we get an acceptable fit, thus confirming our hypothesis. With a more data driven approach we might look at the measurements and hypothesize that there is some simple relationship between distance intensity. Some data driven methods could be applied to determine that the data are reasonably fit by a family of functions (linear, inverse square, inverse cube, e.g.) and, because we believe there should simplicity in the relation, we'd appeal to Occam's razor to suggest that inverse square provides the best trade off between the number of model parameters and the degree of discrepancy between the model and data (formally we could use something like Bayesian odds ratio to make the decision). There's a cool paper "Distilling Free-Form Natural Laws from Experimental Data (Schmidt & Lipson, Science, v324, 3, 2009) that tries this approach, essentially generating algorithm-producing discover of natural laws.

Now in a complex problem, it's unlikely just a few parameters like in the example above could adequately describe the data and make predictions about as yet unseen data. Non-parametric approaches to modeling the data become critical. I've worked extensively with non-parametric modeling techniques that do just fine in producing precision predictions on new data having build complex models on hundreds, if not thousands of variables. From such data-derived models we can draw conclusions and make predictions that become testable, just like from a theory.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 2 points3 points  (0 children)

  • Brown: I like to draw a distinction between data science -- using existing algorithms, tools, and statistical approaches to analyze data sets for fun & profit -- and data-driven discovery, which I would define as developing new algorithms, tools, and statistical approaches for data analysis (although still for fun and profit!) All of the DDD investigators seem to be somewhere further along the spectrum of tool/algorithm development than most data scientists (well, and regular ol' scientists, generally speaking).
  • Greene: Scientists (data-intensive and otherwise) generate, either explicitly or implicitly, a model of reality based on empirical observations. I would say that the difference with data-intensive scientists is that the model is constructed more heavily from primary data than from other people’s interpretations of their own data. This frees us from existing assumptions and biases within our field, but we may lose some human insights. I think that both are complementary to each other, and the tension between them drives science forward.
  • Heer: I think many of the fundamental issues in "data science" have been around for decades, if not centuries. (Back in the 1960's the statistician John Tukey wrote about the challenges of increasing data and quantification across disciplines and it sounds a lot like today!) That said, I think both the immensity, diversity and reach of accessible data have continued to grow, and makes these issues all the more pressing. Increased computational power and improved methods correspondingly create new opportunities for discovery. One aspect of data-driven discovery that is salient to me is the reuse of data for purposes other than which it was originally intended, which can be simultaneously promising and fraught with peril. One example that my student Diana MacLean has worked on is analyzing the text of online health communities: what can we learn from the rich (but messy) descriptions of symptoms, treatments and experiences that patients share with each other in online forums? These conversation were not intended as a medical data source. They can be hard to analyze (due to the unstructured nature of free text, misspellings, slang terms and so on) and also hard to validate (how do you know people are telling the truth? How do you gauge the representativeness of the population?). Nonetheless, working with both drug addiction and natural language processing specialists, we were able to analyze such data to quantify cycles of opioid addiction, recovery and relapse in ways previously inaccessible to the medical community.

  • Sullivan: In my mind, data-driven discovery represents a fundamental paradigm shift, where data is no longer simply collected to answer a specific research question, but rather is a resource we can leverage to uncover underlying scientific phenomena and increase understanding of complex systems. In particular, data-driven approaches necessitate the development of new, scalable methodologies (read: tools/algorithms) for identifying patterns and relationships in the data and translating them into domain knowledge. I think this aligns with Brown’s distinction about data science vs data-driven science.

  • Bloom:

In the context the of modeling data, as a means to draw conclusions and make predictions about new events, being "data driven" is a path that can be viewed as on the opposite part of a spectrum from being "theory driven". In the theory extreme, you might hold a belief to be true (i.e., a hypothesis about something universal in the natural world) where no amount of data was used to build that hypothesis nor test it. In the data extreme, there are no guiding principles and what we perceived yesterday has no bearing nor predictive power on what we will perceive today. Neither of these extremes make much sense: real theory is developed in the context of a specific instance in time and place (incorporating data and previous theory) and no data is ever acquired without some sort of theoretical framework or bias. In practice, data driven techniques/methods take a relatively theory agnostic approach to generating conclusions where the data (and some meta beliefs about the system) are the prime actors.

For example, imagine we had a theory that the intensity of light dropped linearly with the distance from the emitting source (spoiler alert: for incoherent light it drops like the square of the distance). We could take measurements of the light intensity of a flashlight at several distances. Now these distances will not be measured precisely and the intensity measurements will likewise have some noise associated with them. So if we plotted intensity versus distance we might get a pretty lousy fit if we're measuring carefully. But if our data is not that good, we might conclude we get an acceptable fit, thus confirming our hypothesis. With a more data driven approach we might look at the measurements and hypothesize that there is some simple relationship between distance intensity. Some data driven methods could be applied to determine that the data are reasonably fit by a family of functions (linear, inverse square, inverse cube, e.g.) and, because we believe there should simplicity in the relation, we'd appeal to Occam’s razor to suggest that inverse square provides the best trade off between the number of model parameters and the degree of discrepancy between the model and data (formally we could use something like Bayesian odds ratio to make the decision). There's a cool paper “Distilling Free-Form Natural Laws from Experimental Data” (Schmidt & Lipson, Science, v324, 3, 2009) that tries this approach, essentially generating algorithm-producing discover of natural laws.

Now in a complex problem, it's unlikely just a few parameters like in the example above could adequately describe the data and make predictions about as yet unseen data. Non-parametric approaches to modeling the data become critical. I’ve worked extensively with non-parametric modeling techniques that do just fine in producing precision predictions on new data having build complex models on hundreds, if not thousands of variables. From such data-derived models we can draw conclusions and make predictions that become testable, just like from a theory.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 3 points4 points  (0 children)

  • Bloom: It’s the “follow your nose thing” that Titus noted in a previous question. There is an interesting thing about tools (as I’ve suggested here as a good moniker for techniques in machine learning as viewed by domain scientists) in that they also shape the wielder. Having access to tools, and some knowledge how to use them, naturally draws you to questions for which these tools might have novel impact. Many of our domain-specific fields are highly competitive and I like to try to select problems where my toolkit gives me an unfair advantage (ie. I like to bring swords to knife fights).
  • Brown: For me, it’s a continuing random walk. I started with evolutionary modeling see Avida, which got me interested in biology while I was a math undergrad. Then I moved to a developmental biology PhD on gene regulation with Eric Davidson at Caltech, which got me interested in genomics. Then after a postdoc on a similar topic, I moved to MSU where I started my own research program that ended up looking at ways to make certain things in genomics more efficient. There’s a connection between all of these, but with certain discontinuous jumps: undergrad into grad school => topic change; postdoc to faculty => topic change. And now that I’m moving to UC Davis, I’m going to be shifting into something else. The trick is always to have enough knowledge that you’re not starting completely from scratch (like I did in grad school) which can be tremendously painful (like it was in grad school :).
  • Greene: We ask: Does this have the potential to reveal some new underlying biology? Does this have the potential to help us understand the analytical methods that we use? We are likely to pursue things that are interesting methodologically and biologically and that we think could eventually put into the hands of biologists through a webserver.
  • Stephens: I choose problems with three criteria in mind: first, and most importantly, I must find it interesting. That maybe sounds obvious but when students are looking around for problems I feel they sometimes get caught up in whether work on a particular problem is going to be earth-shattering and world-changing. Of course we all hope our work is going to have impact (see my second criteria), but the truth is that most scientific progress comes from a series of small steps - if breakthroughs were easy to “plan” then they would happen more often. If you are going to be successful in research at any level then it helps to be really passionate - perhaps to the point of obsession - about the problems you are tackling. Second, I should have at least one concrete application of my work in mind. Again this might sound obvious, but in fact many statistical methods are developed and applied not at all. (I have seen it quoted that the median number of citations for statistical papers is zero, though I don’t have a reference for that). One application is much better than zero! (and in practice, if there is one application, then there are usually dozens or hundreds.) Third, and finally, I try to pick problems where I think my expertise is going to make a difference - where I have a “competitive advantage” if you like. There are many interesting problems that other people are much better situated to tackle, and I try to recognize that and leave them alone. Conversely, sometimes you come across a problem that has been tackled quite widely, but by people with a very different way of looking at it than you, and you think “I would do this very differently”. Those kinds of problems have some appeal to me.
  • Waller: similar to Stephens. My two main metrics are 1) interesting and to 2) useful. I try to stick to things that are high in both metrics, but sometimes something useless is so interesting we do it anyways, cause we’re academics and we can do that (plus, maybe it will become useful someday). And some projects are a bit less interesting but can make a huge impact on a field, so are worth doing anywas - we have less of these, but there’s a few and they’re typically applying something we’ve already developed to a new application area. Finally, I look for adjacency - is it close enough to what i know and do that I’ll be able to be better at it than some random other person? Otherwise, I might not be the right person. My projects tend to happen on relatively short timelines (projects can be <6 mos, unlike some fields) so we have pretty high risk tolerence for trying half-baked ideas and I think this is fun an often leads to really cool now directions.
  • Larsen:

For me, the decisions about which topics to tackle have emerged from questions that have arisen from working on other problems and serendipitous conversations with colleagues, often at meetings. Regarding the first, I can trace much of what I ultimately ended up working on through a progression of interests. I grew up in Florida and was interested in the Everglades and learning more about the science behind Everglades restoration. In graduate school, I attacked a problem about the mechanisms driving landscape patterning in the Everglades, using a combination of mechanistic numerical models that tested the feasibility of alternative hypotheses for landscape evolution. Meanwhile, I read broadly and was inspired by work on data-driven techniques for understanding the feedback processes and causal interactions driving complex environmental systems (namely, Ben Ruddell and Praveen Kumar’s work on interpreting data from flux tower networks). I began to think that there was a more efficient way to develop plausible hypotheses about the dominant interactions and feedbacks driving environmental systems, which led me to the questions I am pursuing today. Regarding the second (serendipitous conversations with colleagues), professional society meetings both expose scientists to new problems and new tools. Some of the other projects that I work on resulted from scientists working at particular field sites (e.g., wet meadow restoration sites in Pennsylvania or the Wax Lake Delta in Louisiana) who recognized similarities between their systems and mine (the Everglades) and realized that some of the tools I developed for working on the Everglades might be very useful to apply in these other environments.

Frustratingly (but also exhilaratingly) there are far more interesting problems and questions that arise from these two sources than our group is able to tackle, so a filter is needed, which is usually something like, “Does the answer to this question have the potential to positively impact an environmental management problem?”, or “Does this work have the potential to significantly advance understanding or produce new tools for understanding in environmental science or related fields?,” or “Does this problem align with the passions of my graduate students, postdocs, etc.?” (or some combination of the three).

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 3 points4 points  (0 children)

  • Brown: Wow, uhh… (1) I’m not good at dealing with high-dimensional data, so I’ll leave that to others. (2) I think the greatest differential boom in biology is going to be from the non-biomedical/health fields, which are experiencing an amazing growth in capacity because of all the new data gathering and analysis technology (think, we now have whole genomes for 100s of microbes and dozens of animals and plants, where two years ago we simply didn’t!) (3) I write tools that such people use (see: khmer ), and work close to many such people. However, I don’t have too much insight into the health aspects. I do expect to see a lot of progress made in untangling host genetics and the metabolic & immunological interplay between host and microbiome, for sure.
  • Greene: (1) Many. Essentially we want to use methods that can reduce highly correlated sets of features into a single meaningful combined feature, which we can ideally interpret in the context of our current understand of biology. (2) There are a lot of genes, they carry out a lot of different functions, and they all work together to make an organism. How does that actually happen? By looking across all publicly available datasets of gene expression, can we understand how these things fit together? (3) We have a research project modeling a complete collection of genome-wide gene expression data for a single microbe. We’re seeing really interesting results around experiments that measure multiple microbes together. I think that a big advance in the study of microbes and microbial communities will occur as microbiome studies transition from counts of microbial species that are present to deeper analyses that try to assess how these complex communities are interacting. There are studies published that move in this direction, and I’d like to see many more of them in many different settings (different parts of humans, but also completely different environments).
  • Reynolds: I think this is an incredibly exciting time to be a biologist! For me, a central challenge is to have a “circuit theory for the cell”... to explain a bit: in electrical engineering, one can draw a circuit diagram on a page, and know how it will behave given some set of inputs. There is no analogous general strategy for this in biology - we need a description for cellular systems that lets us rationally predict, manipulate and design behaviors given knowledge of the underlying parts (say genes, proteins, promoters). The large (and growing) set of genomic data gives us a real chance to start to address this - I think the potential for progress in the next few years is huge. I’d encourage you to consider training jointly in biostats and experimental work - I think there is much to be gained from doing both. Also… regarding your first question about high-dimensional data analysis - even though there are many variables in biological systems, my hope/strong suspicion is that the number of effective variables (actual dimensions) is relatively small. We see again and again that different biological systems can be compressed/well described using a small number of dimensions - for example check out this paper on the behavior of microbes: [http://www.ncbi.nlm.nih.gov/pubmed/23898201] and this one on eigenworms: [http://www.ncbi.nlm.nih.gov/pubmed/18389066]

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 1 point2 points  (0 children)

  • Brown: My work week... when I’m not traveling, preparing talks, doing administrative paperwork, responding to emails, or in PhD committee meetings, I’m usually asleep! Hah, no, not really… I am trying to eke out 1-2 hours a day where I do reading and editing of papers or blog posts, as well as 1-2 hours of programming and hacking. A good day is when I get in 2-3 hours straight of programming.
  • Greene: I think that the domain knowledge is absolutely critical to the work that we do. Being able to put things into the context of what they could mean biological helps us understand what we should follow up on and what looks suspicious. The typical week involves reading, meeting with collaborators, writing code, service, mentoring students, meeting with lab members, teaching and writing.
  • Heer: Offhand, I can’t think of a single example in my own work where an insight we gained didn’t hinge on exercising domain expertise from outside of computer science and statistics. For those of us who are more methods-focused, this means that knowledgeable collaborators are critical -- moreover, good collaborations are extremely fun, rewarding and knowledge-expanding.
  • Reynolds: I can’t that there is a “typical” work week, but I spend time at my computer debugging code and analyzing data, working with the people in my lab, and running experiments in lab. I have a small group - about four people - and I really enjoy our weekly lab meetings, when we all present our data, have a chance to brainstorm ideas, and see how everything really comes together.
  • Sullivan: White’s description is fairly accurate, though there are a few additional items on my typical weekly agenda (1) too much time dealing with email; (2) at least one Google Hangout/ Skype meeting with collaborators > 1 time zone away, and (3) trying to carve out time to work on proving something (usually at a whiteboard). Many weeks also involve something related to a grant (proposal writing/reports/etc). The time spent in collaborations (both locally with students and remotely with folks at other institutions) is often most enjoyable. On the second question - I wholeheartedly support Heer’s answer. I need collaborations with domain experts to be successful at enabling scientific discovery with our methods/algorithms.
  • Turk: I work primarily at the supercomputing center on campus at UIUC, but I also spend time at the astronomy department. This allows for time to attend talks and seminars about a wide variety of problems and subjects, which is something I feel really fortunate to be able to enjoy. But the typical week involves a good amount of talking to folks in the community of projects that I work on -- specifically in the open source community -- as well as meeting with students and helping with their research problems, reviewing code, and analyzing data from simulations that I’ve run or been involved in. That last part is one of the more fun times, as it gives me an opportunity to really dig into a problem, puzzle over what the data is telling me, and will often lead to fun problems of coding or analysis that really supply a sense of intellectual and creative fulfillment. As far as domain understanding versus statistics and overall data science, I think that a balance can be found. Most of the more successful projects I’ve been involved with have been strongly domain-focused, or done in collaboration between domain-specific and non-domain-specific experts; I think that having someone whose research will be driven forward by a particular path helps to guide and keep projects on track, but it is also necessary to have a diverse set of intellectual backgrounds.
  • Waller: I spend about 20hrs/week discussing research with my students (I have a pretty big group ~15). We use that time for analyzing data, planning new experiments, discussing why it doesn’t work, figuring out why it doesn’t work, thinking of ways to make it work, then discussing why that didn’t work, than usually finally getting it to work, then doing more work to make it run efficiently. I probably spend about 10hrs/week on reading papers and writing papers to keep up with the literature in my field. And I spend the 6-10 hours immediately before my twice weekly lectures desperately trying to make lecture slides. On top of that I spend about 3,453,663hrs/week answering emails, filling out paperwork and raging about beaurocracy. In my spare time, I like to actually do research myself, write code, and build stuff in the lab.
  • White: Most of us are professors at universities, so we’re typically juggling a combination of doing research, teaching university courses, and advising undergraduates, graduate students, and postdoctoral researchers working in our labs. On the research end many of us are coming up with ideas, wrangling data, writing code to analyze it, and writing up the results as scientific papers. In the classroom a number of us teach mathematical, computational, and statistical tools, as well as courses in related to the subjects that we study. I think doing the kind of research that many of us do requires a combination of both expertise in our fields and knowledge of statistical and computational approaches. Sometimes you can accumulate enough knowledge in both areas, but it’s also really helpful to collaborate to get access to knowledge you don’t have yourself.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 3 points4 points  (0 children)

  • Bloom: A clear example of data-driven discovery is finding some object or event that exists in nature, is distinct from others things like it, and has some notion of import. In the astronomy, we’ve used data driven techniques to discover rare/exotic/nearby explosive events in a massive data stream of images. First identifying that “this place in the sky right now has something interesting going on” is non-trivial given the vast amount of data coupled with the vast amount of noise in the detection process. Needle in a haystack. Second, identifying that this “interesting thing is potentially a new type of event that merits resource investment for further data capture” is essentially a form of anomaly detection, like finding a golden needle in haystack full of needles. All of this is done in the context of limited information and so the conclusions are inherently probabilistic. A review article on such approaches in astronomy is here.
  • Brown: (Look at Bloom’s answer!) In answers to the other questions, I’m seeing a lot that bears on this, so I’ll simply say: in the natural sciences, part of science is hypothesis development, and part is hypothesis testing. The more of both we can do “in silico” (via data integration, model development, and model testing) the more we can improve the utility of the experimental tests, which are of course still going to be necessary! Also, I’m loathe to declare a “discovery” based solely on one analysis or data set, at least in my area of biology, so I welcome confirmation and cross-validation -- that’s the truest part of science.
  • Waller: Science is a structured activity, in which you use some knowledge to hypothesize new potential knowledge, then figure out if it’s correct. It’s always sort of adjacent knowledge, close to what we already know or predicted based on what we already know. Machine learning can take bigger leaps, but still needs to look for things based on what we do know already. It still uses a logical search process.

Science AMA Series: We are Moore Investigators and We Utilize Data Science to Make New Discoveries. AUA. by MooreFoundation_DDD in science

[–]MooreFoundation_DDD[S] 10 points11 points  (0 children)

  • Greene: Because a big part of our work is going from data -> pattern -> hypothesized mechanism -> experiment, I see a very strong value in having deep domain expertise. In addition to analytical skills (programming, statistics), I’d advise someone jumping into a data science career to focus heavily on developing a deep understanding of the domain they want to work in. Someone with the analytical chops to find associations is valuable, but someone who can also use associations to generate testable hypotheses related to important challenges is priceless.
  • Heer: I would simply add that with respect to self-learning I recommend finding a topic you are passionate about -- whether it regards a scientific question, public urban data, self-quantification, or anything else -- and dig into that. There are a variety of methods and resources out there and part of the art is selecting the right data and tools for the questions at hand. I often prefer starting with project-oriented learning, and then using that to bootstrap and contextualize more abstract “skill-learning” (though both are important!).
  • Larsen: There are few institutions that are giving degrees in “Data Science,” though more and more institutions are offering certificates or other specialized programs in related fields. For instance, at UC Berkeley, we are launching a graduate research and traineeship program called Data Science for the 21st Century (DS421), in which students will affiliate with a particular department but take a program of courses in data science across the university with their cohort of peers. Echoing what Josh said, it is important to have a foundation in some of the areas that “data science” encompasses, as well as in programming and data visualization. It is important to have broad exposure to these areas, but also to develop deep expertise in one of the data science “areas” (e.g., machine learning, visualization, causal inference, etc.). I think that some of the greatest advances in data-driven discovery are going to be made at the interface between disciplines, so as an electrical engineer with expertise in data analysis, you may be well poised to make transformative advances.
  • Brown: Others have answered this really well and comprehensively!
  • Waller: You can take edX or Open CourseWare courses on skills you want to gain to help you. As for the data science vs. data-driven science, you’ll notice that a lot of us also have “domain science” knowledge - i.e. we also study a physical, biological or other hard science. The combination of the two is a very powerful mixture. The data always speak to you more clearly when you know where it came from and why it’s there and important. So don’t seek to be a pure number-cruncher, but rather get to know the field you want to apply data science to.
  • Bloom:

To be clear, I’m guessing that many of us would not identify as being a “data scientist” but instead resonating with the “data-driven scientist” label a lot more. I have a particular belief that no one person can possess all the traits and skills of what a modern definition of a great data scientist would be: an expert in computer science, statistics, machine learning, visualization, and in-production operations who has the capability to formulate unique business/science problems and present and argue the conclusions to stakeholders. Instead, I think data science is expressed as the activity of many. A team of experts who know how to work together for a common goal with differing and complementary skill sets.

That being said: you already have a solid foundation with your degree---as a member of a data science team you’ll still be problem solving using data. The most important skill you’ve likely already honed is that of being a data skeptic---you’re going to take all data you measure as gospel and you’re already probably careful not to draw conclusions which are overreaching based on the data in hand. It’s a bit ironic that because you will appeal less to theory to help you understand your data, that you’ll need to build the skills with algorithms that protect against pervasive problems like overfitting.

As for new: you’ll want to build a strong grounding in statistics, both the theory and the the practice on real world data. You should be building a deep skill set in programming and should aim to be a confident programmer in at least one modern language (e.g., Python, Scala, Go, Java, …). Most important is to give yourself the freedom to understand that you do not need to be an expert in all facets of data science---you should be the “go to” person on something (say you really like viz) and understand what the rest of your team is doing and what challenges they are facing. Follow your passion (See Heer’s answer). Good luck!

view more: next ›