UoL CS Notes
Menu

Evaluating and Gathering Information Sources

COMP107 Lectures

How to we form our knowledge?

Generally we form our personal collection of facts from sources that we meet in our every day life. This means that a good proportion of the knowledge we might think that we have is composed of myths. This is especially found in news articles.

These articles are generally based in fact in the sense that they reference real journals. The articles covert the unreadable and specialised format of the journal into a more sensationalised and popularised format.

Assessing the Source

Argument from Expert Opinion

When assessing arguments from expert opinion the following must be taken into account:

  • Expertise
    • How credible is the source?
  • Field
    • Is the source expert in the field?
  • Opinion
    • What did the source actually say? When?
  • Trustworthiness
    • Is the use of the source reliable or does the writer have any alternator motive where they might not want to portray the data as it was collected?
  • Consistency
    • Is the source consistent with others?
      • Peer Review

Main Academic Sources

A scholarly paper is a way for expert in the field to communicate their results, typically to other experts in the field.

A popular paper is a communication to the general public about some research. They may not be as rigorous or detailed as a scholarly paper.

Conference Papers

Gatherings for researchers to present and discuss their work, typically related to a particular academic discipline and often held at regular intervals.

Output published in “conference proceedings” in the form of conference papers written by the researchers about their work, and often peer-reviewed

Types of events:

  • Conference
  • Workshop
  • Symposium

You should check the acceptance ratio to check what people think of the paper.

Journal Articles

Academic journals are peer-reviewed periodicals in with research relating to a particular academic discipline is published.

Each issue of a journal contains a collection of articles, each article written by a group of researchers.

There may be special issues on a single topic.

Dependency on Discipline

Computer Science is a dynamic subject, and communication of results, and publication, relies heavily on conferences.

This is compared to other disciplines which focus on publishing in journals.

Therefore

A prestigious Computer Science conference has a very rigorous peer review process. Papers tend to be longer and more exhaustive.

A medical conference, for contrast, will present mainly 300 word abstracts with the intention that a full paper will follow on a journal.

Publication Pipeline

  1. Authors submit paper to conference/journal for peer review.
    • At least two peer reviews reading independently.
    • For journals, they could have many iterations.
  2. If accepted, the paper is revised by the authors and submitted to the conference journal editor.
  3. The paper is processed to bring it into the publisher’s format (typesetting/layout).
  4. The paper is then included in the publisher’s database and possibly published in printed form.
  5. Literature databases collect the bibliographic information from several publishers, and add additional information (references with links, citation index) and link back to publisher for full text of papers.

Publisher Databases and Search Engines

Have a look at the slides for a list of the publishers that the university subscribes to on the library page: discover (uol).

If a journal is included in a journal/proceedings database then there has been a review process in order for it to be accepted. This may mean that it is more trustworthy.

Get into the habit of using Google Scholar to find other papers or number of citations leading to a paper.

The site ArXiv.org hosts pre-prints of papers that are moderated but not peer-reviewed. This allows us to get bleeding edge information on very current topics such as COVID-19. They should be taken in context.

DBs v.s. Search Engines

Literary databases cover a vast number of academic sources but:

  • They do not cover all journals conferences.
  • They do not cover books.
  • They do not cover workshops and similar scientific meetings.
  • They do not cover technical reports and pre-prints.

Web search engines provide much better coverage of all types of publications but:

  • Typically also return a lot of irrelevant material to a query.
  • Leave it to the user to distinguish high quality form low quality material.

CRAAP

This is a useful, multi-platform and multidisciplinary checklist to evaluate sources.

  • Currency
    • When was the information published or posted?
    • Has the information been revised or updated?
    • Is the information current or out-of date for your topic?
    • Are the links functional?
  • Relevance
    • Does the information relate to your topic/answer your question?
    • Who is the intended audience?
    • Is the information at an appropriate level?
    • Have you looked at a variety of sources before determining this is one you will use?
    • Would you be conformable using this source in your paper?
  • Authority
    • Who was involved in the writing of the paper?
    • Are the authors credentials or affiliations given?
    • What are the authors credentials or affiliations?
    • What are the authors qualifications?
    • Are there any contact information?
    • Does the URL reveal anything about the author or source (the domain)?
  • Accuracy
    • Where does the information come from?
    • Is the information supported by evidence?
    • Has the information been reviewed or refereed?
    • Can you verify any of the information by another source or from personal knowledge?
    • Does the language or tone seem biased and free of emotion?
    • Are there spelling, grammar, or other typographical errors?
  • Purpose
    • What is the purpose of the information?
    • Do the authors/sponsors make their intentions or purpose clear?
    • Is the information fact?
    • Does the point of view appear objective and impartial?
    • Are there any biased?

For each point you can give a source a score from one to ten. If the score if above 40 then the source is good or excellent. Bellow 30 makes the source problematic as a source of information.