Feeds:
Posts
Comments

Posts Tagged ‘scientific curation’

Science Has A Systemic Problem, Not an Innovation Problem

Curator: Stephen J. Williams, Ph.D.

    A recent email, asking me to submit a survey, got me thinking about the malaise that scientists and industry professionals frequently bemoan: that innovation has been stymied for some reason and all sorts of convuluted processes must be altered to spur this mythical void of great new discoveries…..  and it got me thinking about our current state of science, and what is the perceived issue… and if this desert of innovation actually exists or is more a fundamental problem which we have created.

The email was from an NIH committee asking for opinions on recreating the grant review process …. now this on the same day someone complained to me about a shoddy and perplexing grant review they received.

The following email, which was sent out to multiple researchers, involved in either NIH grant review on both sides, as well as those who had been involved in previous questionnaires and studies on grant review and bias.  The email asked for researchers to fill out a survey on the grant review process, and how to best change it to increase innovation of ideas as well as inclusivity.  In recent years, there have been multiple survey requests on these matters, with multiple confusing procedural changes to grant format and content requirements, adding more administrative burden to scientists.

The email from Center for Scientific Review (one of the divisions a grant will go to before review {they set up review study sections and decide what section a grant should be  assigned to} was as follows:

Update on Simplifying Review Criteria: A Request for Information

https://www.csr.nih.gov/reviewmatters/2022/12/08/update-on-simplifying-review-criteria-a-request-for-information/

NIH has issued a request for information (RFI) seeking feedback on revising and simplifying the peer review framework for research project grant applications. The goal of this effort is to facilitate the mission of scientific peer review – identification of the strongest, highest-impact research. The proposed changes will allow peer reviewers to focus on scientific merit by evaluating 1) the scientific impact, research rigor, and feasibility of the proposed research without the distraction of administrative questions and 2) whether or not appropriate expertise and resources are available to conduct the research, thus mitigating the undue influence of the reputation of the institution or investigator.

Currently, applications for research project grants (RPGs, such as R01s, R03s, R15s, R21s, R34s) are evaluated based on five scored criteria: Significance, Investigators, Innovation, Approach, and Environment (derived from NIH peer review regulations 42 C.F.R. Part 52h.8; see Definitions of Criteria and Considerations for Research Project Grant Critiques for more detail) and a number of additional review criteria such as Human Subject Protections.

NIH gathered input from the community to identify potential revisions to the review framework. Given longstanding and often-heard concerns from diverse groups, CSR decided to form two working groups to the CSR Advisory Council—one on non-clinical trials and one on clinical trials. To inform these groups, CSR published a Review Matters blog, which was cross-posted on the Office of Extramural Research blog, Open Mike. The blog received more than 9,000 views by unique individuals and over 400 comments. Interim recommendations were presented to the CSR Advisory Council in a public forum (March 2020 videoslides; March 2021 videoslides). Final recommendations from the CSRAC (report) were considered by the major extramural committees of the NIH that included leadership from across NIH institutes and centers. Additional background information can be found here. This process produced many modifications and the final proposal presented below. Discussions are underway to incorporate consideration of a Plan for Enhancing Diverse Perspectives (PEDP) and rigorous review of clinical trials RPGs (~10% of RPGs are clinical trials) within the proposed framework.

Simplified Review Criteria

NIH proposes to reorganize the five review criteria into three factors, with Factors 1 and 2 receiving a numerical score. Reviewers will be instructed to consider all three factors (Factors 1, 2 and 3) in arriving at their Overall Impact Score (scored 1-9), reflecting the overall scientific and technical merit of the application.

  • Factor 1: Importance of the Research (Significance, Innovation), numerical score (1-9)
  • Factor 2: Rigor and Feasibility (Approach), numerical score (1-9)
  • Factor 3: Expertise and Resources (Investigator, Environment), assessed and considered in the Overall Impact Score, but not individually scored

Within Factor 3 (Expertise and Resources), Investigator and Environment will be assessed in the context of the research proposed. Investigator(s) will be rated as “fully capable” or “additional expertise/capability needed”. Environment will be rated as “appropriate” or “additional resources needed.” If a need for additional expertise or resources is identified, written justification must be provided. Detailed descriptions of the three factors can be found here.

Now looking at some of the Comments were very illuminating:

I strongly support streamlining the five current main review criteria into three, and the present five additional criteria into two. This will bring clarity to applicants and reduce the workload on both applicants and reviewers. Blinding reviewers to the applicants’ identities and institutions would be a helpful next step, and would do much to reduce the “rich-getting-richer” / “good ole girls and good ole boys” / “big science” elitism that plagues the present review system, wherein pedigree and connections often outweigh substance and creativity.

I support the proposed changes. The shift away from “innovation” will help reduce the tendency to create hype around a proposed research direction. The shift away from Investigator and Environment assessments will help reduce bias toward already funded investigators in large well-known institutions.

As a reviewer for 5 years, I believe that the proposed changes are a step in the right direction, refocusing the review on whether the science SHOULD be done and whether it CAN BE DONE WELL, while eliminating burdensome and unhelpful sections of review that are better handled administratively. I particularly believe that the de-emphasis of innovation (which typically focuses on technical innovation) will improve evaluation of the overall science, and de-emphasis of review of minor technical details will, if implemented correctly, reduce the “downward pull” on scores for approach. The above comments reference blinded reviews, but I did not see this in the proposed recommendations. I do not believe this is a good idea for several reasons: 1) Blinding of the applicant and institution is not likely feasible for many of the reasons others have described (e.g., self-referencing of prior work), 2) Blinding would eliminate the potential to review investigators’ biosketches and budget justifications, which are critically important in review, 3) Making review blinded would make determination of conflicts of interest harder to identify and avoid, 4) Evaluation of “Investigator and Environment” would be nearly impossible.

Most of the Comments were in favor of the proposed changes, however many admitted that it adds additional confusion on top of many administrative changes to formats and content of grant sections.

Being a Stephen Covey devotee, and just have listened to  The Four Principles of Execution, it became more apparent that issues that hinder many great ideas coming into fruition, especially in science, is a result of these systemic or problems in the process, not at the level of individual researchers or small companies trying to get their innovations funded or noticed.  In summary, Dr. Covey states most issues related to the success of any initiative is NOT in the strategic planning, but in the failure to adhere to a few EXECUTION principles.  Primary to these failures of strategic plans is lack of accounting of what Dr. Covey calls the ‘whirlwind’, or those important but recurring tasks that take us away from achieving the wildly important goals.  In addition, lack of  determining lead and lag measures of success hinder such plans.

In this case a lag measure in INNOVATION.  It appears we have created such a whirlwind and focus on lag measures that we are incapable of translating great discoveries into INNOVATION.

In the following post, I will focus on issues relating to Open Access, publishing and dissemination of scientific discovery may be costing us TIME to INNOVATION.  And it appears that there are systemic reasons why we appear stuck in a rut, so to speak.

The first indication is from a paper published by Johan Chu and James Evans in 2021 in PNAS:

 

Slowed canonical progress in large fields of science

Chu JSG, Evans JA. Slowed canonical progress in large fields of science. Proc Natl Acad Sci U S A. 2021 Oct 12;118(41):e2021636118. doi: 10.1073/pnas.2021636118. PMID: 34607941; PMCID: PMC8522281

 

Abstract

In many academic fields, the number of papers published each year has increased significantly over time. Policy measures aim to increase the quantity of scientists, research funding, and scientific output, which is measured by the number of papers produced. These quantitative metrics determine the career trajectories of scholars and evaluations of academic departments, institutions, and nations. Whether and how these increases in the numbers of scientists and papers translate into advances in knowledge is unclear, however. Here, we first lay out a theoretical argument for why too many papers published each year in a field can lead to stagnation rather than advance. The deluge of new papers may deprive reviewers and readers the cognitive slack required to fully recognize and understand novel ideas. Competition among many new ideas may prevent the gradual accumulation of focused attention on a promising new idea. Then, we show data supporting the predictions of this theory. When the number of papers published per year in a scientific field grows large, citations flow disproportionately to already well-cited papers; the list of most-cited papers ossifies; new papers are unlikely to ever become highly cited, and when they do, it is not through a gradual, cumulative process of attention gathering; and newly published papers become unlikely to disrupt existing work. These findings suggest that the progress of large scientific fields may be slowed, trapped in existing canon. Policy measures shifting how scientific work is produced, disseminated, consumed, and rewarded may be called for to push fields into new, more fertile areas of study.

So the Summary of this paper is

  • The authors examined 1.8 billion citations among 90 million papers over 241 subjects
  • found the corpus of papers do not lead to turnover of new ideas in a field, but rather the ossification or entrenchment of canonical (or older ideas)
  • this is mainly due to older paper cited more frequently than new papers with new ideas, potentially because authors are trying to get their own papers cited more frequently for funding and exposure purposes
  • The authors suggest that “fundamental progress may be stymied if quantitative growth of scientific endeavors is not balanced by structures fostering disruptive scholarship and focusing attention of novel ideas”

The authors note that, in most cases, science policy reinforces this “more is better” philosophy”,  where metrics of publication productivity are either number of publications or impact measured by citation rankings.  However, using an analysis of citation changes occurring in large versus smaller fields, it becomes apparent that this process is favoring the older, more established papers and a recirculating of older canonical ideas.

“Rather than resulting in faster turnover of field paradigms, the massive amounts of new publications entrenches the ideas of top-cited papers.”  New ideas are pushed down to the bottom of the citation list and potentially lost in the literature.  The authors suggest that this problem will intensify as the “annual mass” of new publications in each field grows, especially in large fields.  This issue is exacerbated by the deluge on new online ‘open access’ journals, in which authors would focus on citing the more highly cited literature. 

We maybe at a critical junction, where if many papers are published in a short time, new ideas will not be considered as carefully as the older ideas.  In addition,

with proliferation of journals and the blurring of journal hierarchies due to online articles-level access can exacerbate this problem

As a counterpoint, the authors do note that even though many molecular biology highly cited articles were done in 1976, there has been extremely much innovation since then however it may take a lot more in experiments and money to gain the level of citations that those papers produced, and hence a lower scientific productivity.

This issue is seen in the field of economics as well

Ellison, Glenn. “Is peer review in decline?” Economic Inquiry, vol. 49, no. 3, July 2011, pp. 635+. Gale Academic OneFile, link.gale.com/apps/doc/A261386330/AONE?u=temple_main&sid=bookmark-AONE&xid=f5891002. Accessed 12 Dec. 2022.

Abstract

Over the past decade, there has been a decline in the fraction of papers in top economics journals written by economists from the highest-ranked economics departments. This paper documents this fact and uses additional data on publications and citations to assess various potential explanations. Several observations are consistent with the hypothesis that the Internet improves the ability of high-profile authors to disseminate their research without going through the traditional peer-review process. (JEL A14, 030)

The facts part of this paper documents two main facts:

1. Economists in top-ranked departments now publish very few papers in top field journals. There is a marked decline in such publications between the early 1990s and early 2000s.

2. Comparing the early 2000s with the early 1990s, there is a decline in both the absolute number of papers and the share of papers in the top general interest journals written by Harvard economics department faculty.

Although the second fact just concerns one department, I see it as potentially important to understanding what is happening because it comes at a time when Harvard is widely regarded (I believe correctly) as having ascended to the top position in the profession.

The “decline-of-peer-review” theory I allude to in the title is that the necessity of going through the peer-review process has lessened for high-status authors: in the old days peer-reviewed journals were by far the most effective means of reaching readers, whereas with the growth of the Internet high-status authors can now post papers online and exploit their reputation to attract readers.

Many alternate explanations are possible. I focus on four theories: the decline-in-peer-review theory and three alternatives.

1. The trends could be a consequence of top-school authors’ being crowded out of the top journals by other researchers. Several such stories have an optimistic message, for example, there is more talent entering the profession, old pro-elite biases are being broken down, more schools are encouraging faculty to do cutting-edge research, and the Internet is enabling more cutting-edge research by breaking down informational barriers that had hampered researchers outside the top schools. (2)

2. The trends could be a consequence of the growth of revisions at economics journals discussed in Ellison (2002a, 2002b). In this more pessimistic theory, highly productive researchers must abandon some projects and/or seek out faster outlets to conserve the time now required to publish their most important works.

3. The trends could simply reflect that field journals have declined in quality in some relative sense and become a less attractive place to publish. This theory is meant to encompass also the rise of new journals, which is not obviously desirable or undesirable.

The majority of this paper is devoted to examining various data sources that provide additional details about how economics publishing has changed over the past decade. These are intended both to sharpen understanding of the facts to be explained and to provide tests of auxiliary predictions of the theories. Two main sources of information are used: data on publications and data on citations. The publication data include department-level counts of publications in various additional journals, an individual-level dataset containing records of publications in a subset of journals for thousands of economists, and a very small dataset containing complete data on a few authors’ publication records. The citation data include citations at the paper level for 9,000 published papers and less well-matched data that is used to construct measures of citations to authors’ unpublished works, to departments as a whole, and to various journals.

Inside Job or Deep Impact? Extramural Citations and the Influence of Economic Scholarship

Josh Angrist, Pierre Azoulay, Glenn Ellison, Ryan Hill, Susan Feng Lu. Inside Job or Deep Impact? Extramural Citations and the Influence of Economic Scholarship.

JOURNAL OF ECONOMIC LITERATURE

VOL. 58, NO. 1, MARCH 2020

(pp. 3-52)

So if innovation is there but it may be buried under the massive amount of heavily cited older literature, do we see evidence of this in other fields like medicine?

Why Isn’t Innovation Helping Reduce Health Care Costs?

 
 

National health care expenditures (NHEs) in the United States continue to grow at rates outpacing the broader economy: Inflation- and population-adjusted NHEs have increased 1.6 percent faster than the gross domestic product (GDP) between 1990 and 2018. US national health expenditure growth as a share of GDP far outpaces comparable nations in the Organization for Economic Cooperation and Development (17.2 versus 8.9 percent).

Multiple recent analyses have proposed that growth in the prices and intensity of US health care services—rather than in utilization rates or demographic characteristics—is responsible for the disproportionate increases in NHEs relative to global counterparts. The consequences of ever-rising costs amid ubiquitous underinsurance in the US include price-induced deferral of care leading to excess morbidity relative to comparable nations.

These patterns exist despite a robust innovation ecosystem in US health care—implying that novel technologies, in isolation, are insufficient to bend the health care cost curve. Indeed, studies have documented that novel technologies directly increase expenditure growth.

Why is our prolific innovation ecosystem not helping reduce costs? The core issue relates to its apparent failure to enhance net productivity—the relative output generated per unit resource required. In this post, we decompose the concept of innovation to highlight situations in which inventions may not increase net productivity. We begin by describing how this issue has taken on increased urgency amid resource constraints magnified by the COVID-19 pandemic. In turn, we describe incentives for the pervasiveness of productivity-diminishing innovations. Finally, we provide recommendations to promote opportunities for low-cost innovation.

 

 

Net Productivity During The COVID-19 Pandemic

The issue of productivity-enhancing innovation is timely, as health care systems have been overwhelmed by COVID-19. Hospitals in Italy, New York City, and elsewhere have lacked adequate capital resources to care for patients with the disease, sufficient liquidity to invest in sorely needed resources, and enough staff to perform all of the necessary tasks.

The critical constraint in these settings is not technology: In fact, the most advanced technology required to routinely treat COVID-19—the mechanical ventilator—was invented nearly 100 years ago in response to polio (the so-called iron lung). Rather, the bottleneck relates to the total financial and human resources required to use the technology—the denominator of net productivity. The clinical implementation of ventilators has been illustrative: Health care workers are still required to operate ventilators on a nearly one-to-one basis, just like in the mid-twentieth century. 

High levels of resources required for implementation of health care technologies constrain the scalability of patient care—such as during respiratory disease outbreaks such as COVID-19. Thus, research to reduce health care costs is the same kind of research we urgently require to promote health care access for patients with COVID-19.

Types Of Innovation And Their Relationship To Expenditure Growth

The widespread use of novel medical technologies has been highlighted as a central driver of NHE growth in the US. We believe that the continued expansion of health care costs is largely the result of innovation that tends to have low productivity (exhibit 1). We argue that these archetypes—novel widgets tacked on to existing workflows to reinforce traditional care models—are exactly the wrong properties to reduce NHEs at the systemic level.

Exhibit 1: Relative productivity of innovation subtypes

Source: Authors’ analysis.

Content Versus Process Innovation

Content (also called technical) innovation refers to the creation of new widgets, such as biochemical agents, diagnostic tools, or therapeutic interventions. Contemporary examples of content innovation include specialty pharmaceuticalsmolecular diagnostics, and advanced interventions and imaging.

These may be contrasted with process innovations, which address the organized sequences of activities that implement content. Classically, these include clinical pathways and protocols. They can address the delivery of care for acute conditions, such as central line infections, sepsis, or natural disasters. Alternatively, they can target chronic conditions through initiatives such as team-based management of hypertension and hospital-at-home models for geriatric care. Other processes include hiring staffdelegating labor, and supply chain management.

Performance-Enhancing Versus Cost-Reducing Innovation

Performance-enhancing innovations frequently create incremental outcome gains in diagnostic characteristics, such as sensitivity or specificity, or in therapeutic characteristics, such as biomarkers for disease status. Their performance gains often lead to higher prices compared to existing alternatives.  

Performance-enhancing innovations can be compared to “non-inferior” innovations capable of achieving outcomes approximating those of existing alternatives, but at reduced cost. Industries outside of medicine, such as the computing industry, have relied heavily on the ability to reduce costs while retaining performance.

In health care though, this pattern of innovation is rare. Since passage of the 2010 “Biosimilars” Act aimed at stimulating non-inferior innovation and competition in therapeutics markets, only 17 agents have been approved, and only seven have made it to market. More than three-quarters of all drugs receiving new patents between 2005 and 2015 were “reissues,” meaning they had already been approved, and the new patent reflected changes to the previously approved formula. Meanwhile, the costs of approved drugs have increased over time, at rates between 4 percent and 7 percent annually.

Moreover, the preponderance of performance-enhancing diagnostic and therapeutic innovations tend to address narrow patient cohorts (such as rare diseases or cancer subtypes), with limited clear clinical utility in broader populations. For example, the recently approved eculizimab is a monoclonal antibody approved for paroxysmal nocturnal hemoglobinuria—which effects 1 in 10 million individuals. At the time of its launch, eculizimab was priced at more than $400,000 per year, making it the most expensive drug in modern history. For clinical populations with no available alternatives, drugs such as eculizimab may be cost-effective, pending society’s willingness to pay, and morally desirable, given a society’s values. But such drugs are certainly not cost-reducing.

Additive Versus Substitutive Innovation

Additive innovations are those that append to preexisting workflows, while substitutive innovations reconfigure preexisting workflows. In this way, additive innovations increase the use of precedent services, whereas substitutive innovations decrease precedent service use.

For example, previous analyses have found that novel imaging modalities are additive innovations, as they tend not to diminish use of preexisting modalities. Similarly, novel procedures tend to incompletely replace traditional procedures. In the case of therapeutics and devices, off-label uses in disease groups outside of the approved indication(s) can prompt innovation that is additive. This is especially true, given that off-label prescriptions classically occur after approved methods are exhausted.

Eculizimab once again provides an illustrative example. As of February 2019, the drug had been used for 39 indications (it had been approved for three of those, by that time), 69 percent of which lacked any form of evidence of real-world effectiveness. Meanwhile, the drug generated nearly $4 billion in sales in 2019. Again, these expenditures may be something for which society chooses to pay—but they are nonetheless additive, rather than substitutive.

Sustaining Versus Disruptive Innovation

Competitive market theory suggests that incumbents and disruptors innovate differently. Incumbents seek sustaining innovations capable of perpetuating their dominance, whereas disruptors pursue innovations capable of redefining traditional business models.

In health care, while disruptive innovations hold the potential to reduce overall health expenditures, often they run counter to the capabilities of market incumbents. For example, telemedicine can deliver care asynchronously, remotely, and virtually, but large-scale brick-and-mortar medical facilities invest enormous capital in the delivery of synchronous, in-house, in-person care (incentivized by facility fees).

The connection between incumbent business models and the innovation pipeline is particularly relevant given that 58 percent of total funding for biomedical research in the US is now derived from private entities, compared with 46 percent a decade prior. It follows that the growing influence of eminent private organizations may favor innovations supporting their market dominance—rather than innovations that are societally optimal.

Incentives And Repercussions Of High-Cost Innovation

Taken together, these observations suggest that innovation in health care is preferentially designed for revenue expansion rather than for cost reduction. While offering incremental improvements in patient outcomes, therefore creating theoretical value for society, these innovations rarely deliver incremental reductions in short- or long-term costs at the health system level.

For example, content-based, performance-enhancing, additive, sustaining innovations tend to add layers of complexity to the health care system—which in turn require additional administration to manage. The net result is employment growth in excess of outcome improvement, leading to productivity losses. This gap leads to continuously increasing overall expenditures in turn passed along to payers and consumers.

Nonetheless, high-cost innovations are incentivized across health care stakeholders (exhibit 2). From the supply side of innovation, for academic researchers, “breakthrough” and “groundbreaking” innovations constitute the basis for career advancement via funding and tenure. This is despite stakeholders’ frequent inability to generalize early successes to become cost-effective in the clinical setting. As previously discussed, the increasing influence of private entities in setting the medical research agenda is also likely to stimulate innovation benefitting single stakeholders rather than the system.

Exhibit 2: Incentives promoting low-value innovation

Source: Authors’ analysis adapted from Hofmann BM. Too much technology. BMJ. 2015 Feb 16.

From the demand side of innovation (providers and health systems), a combined allure (to provide “cutting-edge” patient care), imperative (to leave “no stone unturned” in patient care), and profit-motive (to amplify fee-for-service reimbursements) spur participation in a “technological arms-race.” The status quo thus remains as Clay Christensen has written: “Our major health care institutions…together overshoot the level of care actually needed or used by the vast majority of patients.”

Christensen’s observations have been validated during the COVID-19 epidemic, as treatment of the disease requires predominantly century-old technology. By continually adopting innovation that routinely overshoots the needs of most patients, layer by layer, health care institutions are accruing costs that quickly become the burden of society writ large.

Recommendations To Reduce The Costs Of Health Care Innovation

Henry Aaron wrote in 2002 that “…the forces that have driven up costs are, if anything, intensifying. The staggering fecundity of biomedical research is increasing…[and] always raises expenditures.” With NHEs spiraling ever-higher, urgency to “bend the cost curve” is mounting. Yet, since much biomedical innovation targets the “flat of the [productivity] curve,” alternative forms of innovation are necessary.

The shortcomings in net productivity revealed by the COVID-19 pandemic highlight the urgent need for redesign of health care delivery in this country, and reevaluation of the innovation needed to support it. Specifically, efforts supporting process redesign are critical to promote cost-reducing, substitutive innovations that can inaugurate new and disruptive business models.

Process redesign rarely involves novel gizmos, so much as rejiggering the wiring of, and connections between, existing gadgets. It targets operational changes capable of streamlining workflows, rather than technical advancements that complicate them. As described above, precisely these sorts of “frugal innovations” have led to productivity improvements yielding lower costs in other high-technology industries, such as the computing industry.

Shrank and colleagues recently estimated that nearly one-third of NHEs—almost $1 trillion—were due to preventable waste. Four of the six categories of waste enumerated by the authors—failure in care delivery, failure in care coordination, low-value care, and administrative complexity—represent ripe targets for process innovation, accounting for $610 billion in waste annually, according to Shrank.

Health systems adopting process redesign methods such as continuous improvement and value-based management have exhibited outcome enhancement and expense reduction simultaneously. Internal processes addressed have included supply chain reconfiguration, operational redesign, outlier reconciliation, and resource standardization.

Despite the potential of process innovation, focus on this area (often bundled into “health services” or “quality improvement” research) occupies only a minute fraction of wallet- or mind-share in the biomedical research landscape, accounting for 0.3 percent of research dollars in medicine. This may be due to a variety of barriers beyond minimal funding. One set of barriers is academic, relating to negative perceptions around rigor and a lack of outlets in which to publish quality improvement research. To achieve health care cost containment over the long term, this dimension of innovation must be destigmatized relative to more traditional manners of innovation by the funders and institutions determining the conditions of the research ecosystem.

Another set of barriers is financial: Innovations yielding cost reduction are less “reimbursable” than are innovations fashioned for revenue expansion. This is especially the case in a fee-for-service system where reimbursement is tethered to cost, which creates perverse incentives for health care institutions to overlook cost increases. However, institutions investing in low-cost innovation will be well-positioned in a rapidly approaching future of value-based care—in which the solvency of health care institutions will rely upon their ability to provide economically efficient care.

Innovating For Cost Control Necessitates Frugality Over Novelty

Restraining US NHEs represents a critical step toward health promotion. Innovation for innovation’s sake—that is content-based, incrementally effective, additive, and sustaining—is unlikely to constrain continually expanding NHEs.

In contrast, process innovation offers opportunities to reduce costs while maintaining high standards of patient care. As COVID-19 stress-tests health care systems across the world, the importance of cost control and productivity amplification for patient care has become apparent.

As such, frugality, rather than novelty, may hold the key to health care cost containment. Redesigning the innovation agenda to stem the tide of ever-rising NHEs is an essential strategy to promote widespread access to care—as well as high-value preventive care—in this country. In the words of investors across Silicon Valley: Cost-reducing innovation is no longer a “nice-to-have,” but a “need-to-have” for the future of health and overall well-being this country.

So Do We Need A New Way of Disseminating Scientific Information?  Can Curation Help?

We had high hopes for Science 2.0, in particular the smashing of data and knowledge silos. However the digital age along with 2.0 platforms seemed to excaccerbate this somehow. We still are critically short on analysis!



Old Science 1.0 is still the backbone of all scientific discourse, built on the massive amount of experimental and review literature. However this literature was in analog format, and we moved to a more accesible digital open access format for both publications as well as raw data. However as there was a structure for 1.0, like the Dewey decimal system and indexing, 2.0 made science more accesible and easier to search due to the newer digital formats. Yet both needed an organizing structure; for 1.0 that was the scientific method of data and literature organization with libraries as the indexers. In 2.0 this relied on an army mostly of volunteers who did not have much in the way of incentivization to co-curate and organize the findings and massive literature.



The Intenet and the Web is rapidly adopting a new “Web 3.0” format, with decentralized networks, enhanced virtual experiences, and greater interconnection between people. Here we start the discussion what will the move from Science 2.0, where dissemination of scientific findings was revolutionized and piggybacking on Web 2.0 or social media, to a Science 3.0 format. And what will it involve or what paradigms will be turned upside down?

We have discussed this in other posts such as

Will Web 3.0 Do Away With Science 2.0? Is Science Falling Behind?

and

Curation Methodology – Digital Communication Technology to mitigate Published Information Explosion and Obsolescence in Medicine and Life Sciences

For years the pharmaceutical industry has toyed with the idea of making innovation networks and innovation hubs

It has been the main focus of whole conferences

Tales from the Translational Frontier – Four Unique Approaches to Turning Novel Biology into Investable Innovations @BIOConvention #BIO2018

However it still seems these strategies have not worked

Is it because we did not have an Execution plan? Or we did not understand the lead measures for success?

Other Related Articles on this Open Access Scientific Journal Include:

Old Industrial Revolution Paradigm of Education Needs to End: How Scientific Curation Can Transform Education

Analysis of Utilizing LPBI Group’s Scientific Curation Platform as an Educational Tool: New Paradigm for Student Engagement

Global Alliance for Genomics and Health Issues Guidelines for Data Siloing and Sharing

Multiple Major Scientific Journals Will Fully Adopt Open Access Under Plan S

eScientific Publishing a Case in Point: Evolution of Platform Architecture Methodologies and of Intellectual Property Development (Content Creation by Curation) Business Model 

Read Full Post »

Is It Time for the Virtual Scientific Conference?: Coronavirus, Travel Restrictions, Conferences Cancelled

Curator: Stephen J. Williams, PhD.

UPDATED 3/12/2020

To many of us scientists, presenting and attending scientific meetings, especially international scientific conferences, are a crucial tool for disseminating and learning new trends and cutting edge findings occurring in our respective fields.  Large international meetings, like cancer focused meetings like AACR (held in the spring time), AAAS and ASCO not only highlight the past years great discoveries but are usually the first place where breakthroughs are made known to the scientific/medical community as well as the public.  In addition these conferences allow for scientists to learn some of the newest technologies crucial for their work in vendor exhibitions.

During the coronavirus pandemic, multiple cancellations of business travel, conferences, and even university based study abroad programs are being cancelled and these cancellations are now hitting the 2020 Spring and potentially summer scientific/medical conferences.  Indeed one such conference hosted by Amgen in Massachusetts was determined as an event where some attendees tested positive for the virus, and as such, now other attendees are being asked to self quarantine.

Today I received two emails on conference cancellations, one from Experimental Biology in California and another from The Cancer Letter, highlighting other conferences, including National Cancer Coalition Network (NCCN) meetings which had been canceled.

 

Experimental Biology - San Diego 2020 - April 4-7

Dear Stephen,

After thoughtful deliberations, the leaders of the Experimental Biology host societies have made the difficult but necessary decision to cancel Experimental Biology (EB) 2020 set to take place April 4–7 in San Diego, California. We know how much EB means to everyone, and we did not make this decision in haste. The health and safety of our members, attendees, their students, our staff, partners and our communities are our top priority.

As we have previously communicated via email, on experimentalbiology.org and elsewhere, EB leadership has been closely monitoring the spread of COVID-19 (coronavirus disease). Based on the latest guidance from public health officials, the travel bans implemented by different institutions and the state of emergency declared in California less than 48 hours ago, it became clear to us that canceling was the right course of action.

We thank you and the entire EB community for understanding the extreme difficulty of this decision and for your commitment to the success of this conference – from the thousands of attendees to the presenters, exhibitors and sponsors who shared their time, expertise, collaboration and leadership. We deeply appreciate your contributions to this community.

What Happens Next?

Everyone who has registered to attend the meeting will receive a full registration refund within the next 45 days. Once your registration cancellation is processed, you will receive confirmation in a separate email. You do not need to contact anyone at EB or your host society to initiate the process. Despite the cancellation of the meeting, we are pleased to tell you that we will publish abstracts in the April 2020 issue of The FASEB Journal as originally planned. Please remember to cancel any personal arrangements you’ve made, such as travel and housing reservations. 

We ask for patience as we evaluate our next steps, and we will alert you as additional information becomes available please see our FAQs for details.

And in The Cancer Letter

Coronavirus vs. oncology: Meeting cancellations, travel restrictions, fears about drug supply chain

By Alexandria Carolan

NOTE: An earlier version of this story was published March 4 on the web and was updated March 6 to include information about restricted travel for employees of cancer centers, meeting cancellations, potential disruptions to the drug supply chain, and funds allocated by U.S. Congress for combating the coronavirus.

Further updates will be posted as the story develops.

Forecasts of the inevitable spread of coronavirus can be difficult to ignore, especially at a time when many of us are making travel plans for this spring’s big cancer meetings.

The decision was made all the more difficult earlier this week, as cancer centers and at least one biotechnology company—Amgen—implemented travel bans that are expected to last through the end of March and beyond. The Cancer Letter was able to confirm such travel bans at Fred Hutchinson Cancer Research Center, MD Anderson Cancer Center, and Dana-Farber Cancer Institute.

Meetings are getting cancelled in all fields, including oncology:

The National Comprehensive Cancer Network March 5 postponed its 2020 annual conference of about 1,500 attendees March 19-22 in Orlando, citing precautions against coronavirus.

“The health and safety of our attendees and the patients they take care of is our number one concern,” said Robert W. Carlson, chief executive officer of NCCN. “This was an incredibly difficult and disappointing decision to have to make. However, our conference attendees work to save the lives of immunocompromised people every day. Some of them are cancer survivors themselves, particularly at our patient advocacy pavilion. It’s our responsibility, in an abundance of caution, to safeguard them from any potential exposure to COVID-19.”

UPDATED 3/12/2020

And today the AACR canceled its yearly 2020 Meeting (https://www.aacr.org/meeting/aacr-annual-meeting-2020/coronavirus-information/)

The American Association for Cancer Research (AACR) Board of Directors has made the difficult decision, after careful consideration and comprehensive evaluation of currently available information related to the novel coronavirus (COVID-19) outbreak, to terminate the AACR Annual Meeting 2020, originally scheduled for April 24-29 in San Diego, California. A rescheduled meeting is being planned for later this year.

The AACR has been closely monitoring the rapidly increasing domestic and worldwide developments during the last several weeks related to COVID-19. This evidence-based decision was made after a thorough review and discussion of all factors impacting the Annual Meeting, including the U.S. government’s enforcement of restrictions on international travelers to enter the U.S.; the imposition of travel restrictions issued by U.S. government agencies, cancer centers, academic institutions, and pharmaceutical and biotech companies; and the counsel of infectious disease experts. It is clear that all of these elements significantly affect the ability of delegates, speakers, presenters of proffered papers, and exhibitors to participate fully in the Annual Meeting.

The health, safety, and security of all Annual Meeting attendees and the patients and communities they serve are the AACR’s highest priorities. While we believe that the decision to postpone the meeting is absolutely the correct one to safeguard our meeting participants from further potential exposure to the coronavirus, we also understand that this is a disappointing one for our stakeholders. There had been a great deal of excitement about the meeting, which was expected to be the largest ever AACR Annual Meeting, with more than 7,400 proffered papers, a projected total of 24,000 delegates from 80 countries and more than 500 exhibitors. We recognize that the presentation of new data, exchange of information, and opportunities for collaboration offered by the AACR Annual Meeting are highly valued by the entire cancer research community, and we are investigating options for rescheduling the Annual Meeting in the near future.

We thank all of our stakeholders for their patience and support at this time. Additional information regarding hotel reservation cancellations, registration refunds, and meeting logistics is available on the FAQ page on the AACR website. We will announce the dates and location of the rescheduled AACR Annual Meeting 2020 as soon as they are confirmed. Our heartfelt sympathies go out to everyone impacted by this global health crisis.

However,  according to both Dr. Fauci and Dr. Scott Gottlieb (former FDA director)  the outbreak may revisit the US and the world in the fall (see https://www.cnbc.com/2020/03/04/were-losing-valuable-time-ex-fda-chief-says-of-coronavirus-spread.html)  therefore these meetings may be cancelled for the whole year.

Is It Time For the Virtual (Real-Time) Conference?

Readers of this Online Access Journal are familiar with our ongoing commitment to open science and believe that forming networks of scientific experts in various fields using a social strategy is pertinent to enhancing the speed, reproducibility and novelty of important future scientific/medical discoveries.  Some of these ideas are highlighted in the following articles found on this site:

Scientific Curation Fostering Expert Networks and Open Innovation: Lessons from Clive Thompson and others

Old Industrial Revolution Paradigm of Education Needs to End: How Scientific Curation Can Transform Education

Twitter is Becoming a Powerful Tool in Science and Medicine

e-Scientific Publishing: The Competitive Advantage of a Powerhouse for Curation of Scientific Findings and Methodology Development for e-Scientific Publishing – LPBI Group, A Case in Point

Reconstructed Science Communication for Open Access Online Scientific Curation

In addition, we understand the importance of communicating the latest scientific/medical discoveries in an open and rapid format, accessible over the social media platforms.  To this effect we have developed a methodology for real time conference coverage

see  Press and Conference Coverage

at  https://pharmaceuticalintelligence.com/press-coverage/

AND

The Process of Real Time Coverage using Social Media

at https://pharmaceuticalintelligence.com/press-coverage/part-one-the-process-of-real-time-coverage-using-social-media/

Using these strategies we are able to communicate, in real time, analysis of conference coverage for a multitude of conferences.

Has technology and social media platforms now have enabled our ability to rapidly communicate, in a more open access platform, seminal discoveries and are scientists today amenable to virtual type of meetings including displaying abstracts using a real-time online platform?

Some of the Twitter analytics we have curated from such meetings show that conference attendees are rapidly adopting such social platforms to communicate with their peers and colleagues meeting notes.

Statistical Analysis of Tweet Feeds from the 14th ANNUAL BIOTECH IN EUROPE FORUM For Global Partnering & Investment 9/30 – 10/1/2014 • Congress Center Basel – SACHS Associates, London

Word Associations of Twitter Discussions for 10th Annual Personalized Medicine Conference at the Harvard Medical School, November 12-13, 2014

Comparative Analysis of the Level of Engagement for Four Twitter Accounts: @KDNuggets (Big Data) @GilPress @Forbes @pharma_BI @AVIVA1950

Twitter Analytics on the Inside 3DPrinting Conference #I3DPConf

 

Other Twitter analyses of Conferences Covered by LPBI in Real Time have produced a similar conclusion: That conference attendees are very engaged over social media networks to discuss, share, and gain new insights into material presented at these conferences, especially international conferences.

And although attracting international conferences is lucrative to many cities, the loss in revenue to organizations, as well as the loss of intellectual capital is indeed equally as great.  

Maybe there is room for such type of conferences in the future, and attending by a vast more audience than currently capable. And perhaps the #openscience movement like @MozillaScience can collaborate with hackathons to produce the platforms for such an online movement of scientific conferences as a Plan B.

Other articles on Real Time Conference Coverage in the Online Open Access Journal Include:

Innovations in electronic Scientific Publishing (eSP): Case Studies in Marketing eContent, Curation Methodology, Categories of Research Functions, Interdisciplinary conceptual innovations by Cross Section of Categories, Exposure to Frontiers of Science by Real Time Press coverage of Scientific Conferences

Real Time Coverage and eProceedings of Presentations on 11/16 – 11/17, 2016, The 12th Annual Personalized Medicine Conference, HARVARD MEDICAL SCHOOL, Joseph B. Martin Conference Center, 77 Avenue Louis Pasteur, Boston

Tweets by @pharma_BI and by @AVIVA1950: Real Time Coverage and eProceedings of The 11th Annual Personalized Medicine Conference, November 18-19, 2015, Harvard Medical School

REAL TIME Cancer Conference Coverage: A Novel Methodology for Authentic Reporting on Presentations and Discussions launched via Twitter.com @ The 2nd ANNUAL Sachs Cancer Bio Partnering & Investment Forum in Drug Development, 19th March 2014 • New York Academy of Sciences • USA

Search Results for ‘Real Time Conference’

Read Full Post »

Old Industrial Revolution Paradigm of Education Needs to End: How Scientific Curation Can Transform Education

Curator: Stephen J. Williams, PhD.

Dr. Cathy N. Davidson from Duke University gives a talk entitled: Now You See It.  Why the Future of Learning Demands a Paradigm Shift

In this talk, shown below, Dr. Davidson shows how our current education system has been designed for educating students for the industrial age type careers and skills needed for success in the Industrial Age and how this educational paradigm is failing to prepare students for the challenges they will face in their future careers.

Or as Dr. Davidson summarizes

Designing education not for your past but for their future

As the video is almost an hour I will summarize some of the main points below

PLEASE WATCH VIDEO

Summary of talk

Dr. Davidson starts the talk with a thesis: that Institutions tend to preserve the problems they were created to solve.

All the current work, teaching paradigms that we use today were created for the last information age (19th century)

Our job to to remake the institutions of education work for the future not the one we inherited

Four information ages or technologies that radically changed communication

  1. advent of writing: B.C. in ancient Mesopotamia allowed us to record and transfer knowledge and ideas
  2. movable type – first seen in 10th century China
  3. steam powered press – allowed books to be mass produced and available to the middle class.  First time middle class was able to have unlimited access to literature
  4. internet- ability to publish and share ideas worldwide

Interestingly, in the early phases of each of these information ages, the same four complaints about the new technology/methodology of disseminating information was heard

  • ruins memory
  • creates a distraction
  • ruins interpersonal dialogue and authority
  • reduces complexity of thought

She gives an example of Socrates who hated writing and frequently stated that writing ruins memory, creates a distraction, and worst commits ideas to what one writes down which could not be changed or altered and so destroys ‘free thinking’.

She discusses how our educational institutions are designed for the industrial age.

The need for collaborative (group) learning AND teaching

Designing education not for your past but for the future

In other words preparing students for THEIR future not your past and the future careers that do not exist today.

In the West we were all taught to answer silently and alone.  However in Japan, education is arranged in the han or group think utilizing the best talents of each member in the group.  In Japan you are arranged in such groups at an early age.  The concept is that each member of the group contributes their unique talent and skill for the betterment of the whole group.  The goal is to demonstrate that the group worked well together.

see https://educationinjapan.wordpress.com/education-system-in-japan-general/the-han-at-work-community-spirit-begins-in-elementary-school/ for a description of “in the han”

In the 19th century in institutions had to solve a problem: how to get people out of the farm and into the factory and/or out of the shop and into the firm

Takes a lot of regulation and institutionalization to convince people that independent thought is not the best way in the corporation

keywords for an industrial age

  • timeliness
  • attention to task
  • standards, standardization
  • hierarchy
  • specialization, expertise
  • metrics (measures, management)
  • two cultures: separating curriculum into STEM versus artistic tracts or dividing the world of science and world of art

This effort led to a concept used in scientific labor management derived from this old paradigm in education, an educational system controlled and success measured using

  • grades (A,B,C,D)
  • multiple choice tests

keywords for our age

  • workflow
  • multitasking attention
  • interactive process (Prototype, Feedback)
  • data mining
  • collaboration by difference

Can using a methodology such as scientific curation affect higher education to achieve this goal of teaching students to collaborate in an interactive process using data mining to create a new workflow for any given problem?  Can a methodology of scientific curation be able to affect such changes needed in academic departments to achieve the above goal?

This will be the subject of future curations tested using real-world in class examples.

However, it is important to first discern that scientific content curation takes material from Peer reviewed sources and other expert-vetted sources.  This is unique from other types of content curation in which take from varied sources, some of which are not expert-reviewed, vetted, or possibly ‘fake news’ or highly edited materials such as altered video and audio.  In this respect, the expert acts not only as curator but as referee.  In addition, collaboration is necessary and even compulsory for the methodology of scientific content curation, portending the curator not as the sole expert but revealing the CONTENT from experts as the main focus for learning and edification.

Other article of note on this subject in this Open Access Online Scientific Journal include:

The above articles will give a good background on this NEW Conceived Methodology of Scientific Curation and its Applicability in various areas such as Medical Publishing, and as discussed below Medical Education.

To understand the new paradigm in medical communication and the impact curative networks have or will play in this arena please read the following:

Scientific Curation Fostering Expert Networks and Open Innovation: Lessons from Clive Thompson and others

This article discusses a history of medical communication and how science and medical communication initially moved from discussions from select individuals to the current open accessible and cooperative structure using Web 2.0 as a platform.

 

Read Full Post »

Analysis of Utilizing LPBI Group’s Scientific Curation Platform as an Educational Tool: New Paradigm for Student Engagement

Author: Stephen J. Williams, Ph.D.

 

 

Use of LBPI Platform for Educational Purposes

Goal:  to offer supplemental information for student lessons in an upper level Biology course on Cell Signaling and Cell Motility with emphasis on disease etiology including cancer, neurological disease, and cardiovascular disease.

Course:  Temple University Department of Biology course Cell Signaling and Motility Spring semester 2019. Forty five students enrolled.

Methodology:  Each weekly lesson was presented to students as a PowerPoint presentation.  After each lesson the powerpoint presentation was originally meant to be disseminated to each class-registered student on the students Canvas account.  Canvas is a cloud based Learning Management Software developed by educational technology company Salt Lake City, Utah company Infrastructure, Inc.  According to rough figures, Canvas® charges a setup fee and at least $30 per user (for a university the size of Temple University: 55,000 students at $30 each = 1.6 million a semester for user fees only).

As a result of a technical issue with uploading the first week lesson on this system, I had informed the class that, as an alternative means, class presentation notes and lectures will be posted on the site www.pharmaceuticalintelligence.com as a separate post and searchable on all search engines including Google, Twitter, Yahoo, Bing, Facebook etc. In addition, I had informed the students that supplemental information, from curated posts and articles from our site, would be added to the class lecture post as supplemental information they could use for further reading on the material as well as helpful information and reference for class projects.

The posted material was tagged with #TUBiol3373 (university abbreviation, department, course number) and disseminated to various social media platforms using our system.  This allowed the students to enter #TUBiol3373 in any search engine to easily find their lecture notes and supplemental information.

This gave students access to lectures on a mobile platform which was easily discoverable due to our ability to do search engine optimization. (#TUBiol3373 was among the first search results on most popular search engines).

From a technical standpoint,  the ease at which posts of this nature can be made as well as the ease of including links to full articles as references as well as media has been noted.  Although students seem to navigate the Canvas software with ease, they had noticed many professors have issues or problems with using this software, especially with navigating the software for their needs.   LBPI’s platform is an easily updated, accessible, and extensive knowledge system which can alleviate many of these technical issues and provide the added value of incorporating media based instructional material as well as downloadable file and allow the instructor ability to expound on the presented material with commentary.  In addition due to the social nature of the platform, feedback can be attained by use of curated site statistics and commentary sections as well as online surveys.

 

Results

After the first week, all 45 students used LBPI platform to access these lecture notes with 17 out of 45 continuing to refer to the site during every week (week 1-4) to the class notes.  This was evident from our site statistics as well as number of downloads of the material.  The students had used the #TUBIol3373 and were directed to the site mainly from search engines Google and Yahoo.  In addition, students had also clicked on the links corresponding to supplemental information which I had included, from articles on our site.  In addition, because of the ability to incorporate media on our site, additional information including instructional videos and interviews were included in lecture posts, and this material was easily updated on the instructor’s side.

Adoption of the additional material from our site was outstanding, as many students had verbally said that the additional material was very useful in their studies.  This was also evidenced by site statistics owing to the secondary clicks made from the class lecture post going to additional articles, some not even included as links on the original post.

In addition, and  more important, students had incorporated many of the information from the additional site articles posted and referenced in their class group projects.  At end of semester a survey was emailed to each student  to assess the usefulness of such a teaching strategy. Results of the polling are shown below.

Results from polling of students of #TUBiol3373 “Cell Signaling & Motility” Class

Do you find using a web based platform such as a site like this an easier communication platform for posting lecture notes/added information than a platform like Canvas®? (5 votes)

Answer Votes Percent  
Yes 2 40%  
Somewhat but could use some improvement 2 40%  
No 1 20%  
Did not use web site 0 0%  

 

Do you find using an open access, curated information platform like this site more useful than using multiple sources to find useful extra study/presentation materials? (6 votes)

Answer Votes Percent  
Yes 5 83%  
No 1 17%  

 

Did you use the search engine on the site (located on the top right of the home page) to find extra information on topics for your presentations/study material? (5 votes)

Answer Votes Percent  
Yes 4 67%  
No 1 17%  
Did not use web site 1 17%  

 

Were you able to easily find the supplemental information for each lecture on search engines like Google/Yahoo/Bing/Twitter using the hashtag #TUBiol3373? (6 votes)

Answer Votes Percent  
Yes I was able to find the site easily 4 67%  
No 1 17%  
Did not use a search engine to find site, went directly to site 1 17%  
Encountered some difficulty 0 0%  
Did not use the site for supplemental or class information 0 0%  

 

How did you find the supplemental material included on this site above the Powerpoint presented material for each of the lectures? (7 votes)

Answer Votes Percent  
Very Useful 4 57%  
Did not use supplemental information 2 29%  
Somewhat Useful 1 14%  
Not Useful 0 0%  

How many times did you use the information on this site (https://www.pharmaceuticalintelligence.com) for class/test/project preparation? (7 votes)

Answer Votes Percent  
Frequently 3 43%  
Sparingly 2 29%  
Occasionally 1 14%  
Never 1 14%  

 

 

 

 

 

 

 

Views of #TUBiol3373 lessons/posts on www.pharmaceuticalintelligence.com                    

 

Lesson/Title Total # views # views 1st day # views 2nd day % views day 1 and 2 % views  after 1st 2 days
Lesson 1 AND 2 Cell Signaling & Motility: Lessons, Curations and Articles of reference as supplemental information: #TUBiol3373 60 27 15 93% 45%
Lesson 3 Cell Signaling And Motility: G Proteins, Signal Transduction: Curations and Articles of reference as supplemental information: #TUBiol3373 56 12 11 51% 93%
Lesson 4 Cell Signaling And Motility: G Proteins, Signal Transduction: Curations and Articles of reference as supplemental information: #TUBiol3373 37 17 6 48% 31%
Lesson 5 Cell Signaling And Motility: Cytoskeleton & Actin: Curations and Articles of reference as supplemental information: #TUBiol3373 13 6 2 17% 15%
Lesson 8 Cell Signaling and Motility: Lesson and Supplemental Information on Cell Junctions and ECM: #TUBiol3373 16 8 2 22% 13%
Lesson 9 Cell Signaling: Curations and Articles of reference as supplemental information for lecture section on WNTs: #TUBioll3373 20 10 3 28% 15%
Curation of selected topics and articles on Role of G-Protein Coupled Receptors in Chronic Disease as supplemental information for #TUBiol3373 19 11 2 28% 13%
Lesson 10 on Cancer, Oncogenes, and Aberrant Cell Signal Termination in Disease for #TUBiol3373 21 10 2 26% 20%
Totals 247 69 46 31% 62%
           

 

Note: for calculation of %views on days 1 and 2 of posting lesson and supplemental material on the journal; %views day1 and 2 = (#views day 1 + #views day 2)*100/45 {45 students in class}

For calculation of %views past day 1 and 2 = (total # views – day1 views – day2 views) * 100/45

For calculation in total column last two columns were divided by # of students (45) and # of posts (8)

 

Overall class engagement was positive with 31% of students interacting with the site during the course on the first two days after posting lessons while 61% of students interacted with the site during the rest of the duration of the course.  The higher number of students interacting with the site after the first two days after lecture and posting may be due to a higher number of students using the posted material for study for the test and using material for presentation purposes.

Engagement with the site for the first two days post lecture ranged from 93% engagement to 22% engagement.  As the class neared the first exam engagement with the site was high however engagement was lower near the end of the class period potentially due to the last exam was a group project and not a written exam.  Students appeared to engage highly with the site to get material for study for the written exam however there still was significant engagement by students for purposes of preparation for oral group projects.  Possibly engagement with the site post 2 days for the later lectures could be higher if a written exam was also given towards the end of the class as well.  This type of analysis allows the professor to understand the level of class engagement week by week.

The results of post-class polling confirm some of the conclusions on engagement.  After the final grades were given out all 45 students received an email with a link to the poll.  Of the 45 students emailed, there were 20 views of the poll with 5-7 answers per question.  Interestingly, most answers were positive on the site and the use of curated material for learning and a source of research project material.   It was very easy finding the posts using the #classname and most students used Google to find the material, which was at the top of Google search results.  Not many students used Twitter or other search engines.  Some went directly to the site.  A majority (71%) found the material useful or somewhat useful for their class presentations and researching topics.

Read Full Post »

%d bloggers like this: