Evaluation

In this reading, we’ll look at some of the ways you can evaluate books and articles without killing yourself.  The main thing to remember is that “easiest” and “most convenient” are not evaluation categories.

BOOKS

Why do you have to evaluate books found in a university library?  Aren’t they all good?

Unfortunately, no.  University libraries work to buy the best books they can.  However, information ages: new data renders older theories and interpretations useless, and well-known  authors, such as the late Stephen Ambrose, plagiarize.  University libraries also buy books for courses on topics such as pseudoscience.  These books are good sources only if you are writing a paper on pseudoscience.

You need to evaluate books for other reasons as well.  You need to determine if the book is relevant for your needs,  if it is current enough,  if it contains quality information (particularly important in these days of easy self-publication) and if it’s the right kind of source (scholarly/popular and primary/secondary) for your assignment.

You may also want to evaluate a book to decide if it’s worth spending your time on it.  Reading a book is a definite time commitment, and most people are too busy to read books that a.  they don’t need for class or b.  they don’t want to read for pleasure.

IS THE BOOK SCHOLARLY, POPULAR OR TRADE?

1.  COVERS & TITLE

Check out the cover and title.  You’re not supposed to judge a book by its cover, BUT the cover plus the title will often give a hint as to whether it’s popular or scholarly. 

Probably Scholarly

  • Cover is plain, or uses a piece of art or a photograph related to the subject of the book
  • Title suggests a work aimed at an audience of specialists.  For example:

    Example of a scholarly book cover

    Example of a scholarly book cover

Probably popular

  • Covers have bright interesting art of different types in order to attract buyers.
  • Non-fiction titles suggest a work that will interest readers with a variety of interests and backgrounds.

NOTE:  It can be very difficult to tell if history books are popular or scholarly.  As a general rule, the scholarly books will have significantly more notes and references and more extensive use of primary sources.   Sources in a foreign language are less likely to be translated.

Popular book cover

Popular book cover

Possibly trade

  • Covers can look like popular or scholarly, so you need to check the title.
  • Title should contain a hint that the work has a practical, how-to orientation

    Trade book cover

    Trade book cover

DON’T RELY ON TITLE/COVER ALONE  

The book cover shown below is primary and scholarly.

Cover of a book that looks popular, but is scholarly & popular.Courtesy of Amazon.

Cover of a book that looks popular, but is scholarly &  primary.
Courtesy of Amazon.

2.  CHECK OUT THE AUTHORS

  • Do they have a degree in the field that they’re writing about?
  • Are they affiliated with a university, museum, or other reputable group?
  • What is their reputation in their field?  (You may need to use a search engine to find out)

Books with authors who meet all or most of the criteria above are most likely scholarly.  Popular books are often written by journalists and others without academic credentials.   Some popular books are written by experts who are presenting their work in an easily understandable fashion and some journalists write books with the rigor expected of those with extensive academic credentials.  In the last two cases, you need to look at other clues besides author to make your determination.  When in doubt, check with your instructor or a librarian.

You’re probably wondering how you’re supposed to find all of this information.   (Yes, you are expected to track down this kind of information.)
  • Find and read:  forewords, afterwords, cover blurbs,  etc. – they often have useful information.  This is why you need the whole book even if you don’t read all of it.
  • Check the editorial reviews on Barnes and Noble  (they usually have the best editorial reviews) or Amazon.
  • Barnes and Noble editorial reviews often give author affiliation – click on the Overview tab.  (You may need to scroll down.)
    Example of author information from Barnes & Noble.

    Example of author information from Barnes & Noble. Click to enlarge.
    Courtesy Barnes & Noble.

     

  • You can look at reader reviews, but you must be very careful using them.  Drop the one  star and five star reviews.  Look for the two – four star reviews that say specifically why they did or didn’t like the book AND that give examples from the book.
  • Read book reviews.  Academic Search Premier and JSTOR are good places to look for book reviews.  Most subject databases will also have reviews of books in those subject areas.
  • You can ask a professor.  Just be aware that they have their biases too, so check with the one who will be doing the grading.
  • You can try Wikipedia or do a search, but then you have to be concerned about the quality of the information that you find.
If the author is an academic and/or has scholarly credentials, the book is probably scholarly. Even if popular, it is probably a well done book.
3.  CHECK OUT THE PUBLISHERS
 Check out the publisher of a book.  Books are usually scholarly if they are published by:
  •  A university press.  University presses usually include the word “university” in their name.
    For example:  Oxford University Press, University of Chicago Press, etc.
  • A professional organization.  The words “association,”  “society,”  “institute,” and related terms will usually appear in the name.
    For example: the  Modern Language Association and the Geological Society of America.
  • Trade books are often published by professional organizations and some specialty publishers.
  • There are also some important publishing companies that specialize in academic books.
    For example:  Wiley, Elsevier, ABC-CLIO, and Routledge.
  • In addition to specialty publishers, many of the large general publishing companies, such as HarperCollins, have an academic branch.

4.  LOOK AT THE FRONT MATTER

Front matter is the stuff before you get to the actual book content.  If your book includes a foreward, introduction, prologue, prolegomena, etc. that discusses that authors and their work AND there is a detailed table of contents,  this is a strong indication that the book is scholarly.
5.  LOOK AT THE BACK MATTER
The stuff at the back of the book, after the main content, is called back matter.  Good indicators of a scholarly book include:
  • afterword – this is similar to a foreword, but is at the end of the book.  Often, an author will give more information as to why s/he wrote the book the way they did.
  • references, works cited or bibliography
    NOTE:  for history books and older books in all field, if there is no bibliography or reference list, check for footnotes in the body of the book that include content notes and bibliographic information (citations).
  • appendices (singular = appendix).  An appendix contains additional  information that supports the author’s thesis, but doesn’t really fit into the main body of the book.
  • an extensive index.  Popular books frequently have a fairly short index that links to the major topics.  In most scholarly books,  you can find page numbers for a large number of small, sub-topics as well.
  • maps or illustrations (these can also be listed in the front matter – occasionally, they are part of the front matter, they can also be in the body of the book) Are maps, illustrations, photos, etc. good quality and well labeled?
Remember that these criteria are indicators.  The more of the indicators listed above  a book has, the more likely it is scholarly.  The fewer the indicators,  the more likely the book is popular.  Trade books should have a  practical, how-to orientation.
OTHER EVALUATION CRITERIA
Once you’ve determined if a book is popular, scholarly or trade, you then need to look at the other evaluation criteria:  Is the book relevant?  Objective?  Current enough for your purposes? Credible?  See the table at the end of the reading for more information.
EVALUATING ARTICLES

You evaluate articles the same way you do everything else:  you look at relevance, quality, currency, & objectivity.    In fact, articles are the easiest thing to evaluate, with one big IF.

The easy part is that most articles make it easy for you to find the information you need.  The IF  is:  IF you use the right kind of article.

What’s the right kind of article?  The right kind of article is one that gives you the information you need to do a proper evaluation.  Where do you find these articles?

  • library article databases
  • scholarly search engines (Google Scholar,Pubmed)
    • these search engines find sources other than journal articles.  You must be able to tell the difference.
    • Bing, Yahoo, regular Google, Duckduckgo, etc. are not scholarly search engines.
  • the general web – if you feel like wasting time.
    •  Yes, you will occasionally get lucky.
    • If you use really technical keywords and search Google, it assumes you’re confused and you really meant ot search Google Scholar. Because of that, it may pop out some decent – good articles on the top of the search results page.   If you click the links, you’ll go to Google Scholar, so you might as well start there.
  • Often, the easy way to develop a bibliography of good articles is to check a specialty encyclopedia (not World Book or the Encyclopedia Britannica) or a scholarly book on the subject.
  • Finding sources is one of the legitimate uses of Wikipedia.  One of the signs of a good Wikipedia article is that it has a good reference list and notes.  However, you must still look at the source listed by Wikipedia and decide if it’s a.)  an article and b.) a good source.  So you might as well start with library article databases.  We’ll look at finding articles later in the course.

Finding your article in a good source is an easy way to get a good article.  However, no matter how good the source, you still need to evaluate the article.  You need to make sure that the article is relevant, that it fits your needs, and that the quality is good.  Most articles in library databases or Google Scholar are good, but bad ones creep in occasionally.

Relevancy – Yes, you can make almost any article seem relevant.  Having said that, this is not a game where you try to beat, pound, pry, stuff, strangle  and otherwise manipulate an article into relevancy.  The idea is to start at the right place and end up with a directly relevant, high quality article.

Fits your needs – “Fits your needs” does not mean:  the first thing you find that’s more or less or the subject.  “Fits your needs” means it fits what you need for the assignment you’re doing.   It means you use a scholarly journal article when asked for one instead of trying to sneak a Wikipedia article past your instructor.  It means following the assignment instructions.  It means   asking for help and clarification as needed.

Good quality –  Good quality means that when you go through the evaluation checklist, your article meets all or most of the criteria.

You use the same checklist for books, articles and web sources; however, you focus on slightly different things.  For example:  with books, you look at publisher; for articles you look at the magazine, newspaper, journal or web source.  For all sources, you look at the authors and their affiliation.  If there aren’t identifiable authors, this is usually a sign to stay away from the source.   (Exception:  some sources, especially government and organization documents – online or print – have corporate authors; that is, the organization acts as the author.  Also, some magazine and newspaper articles are written by staff writers and may not have named authors.)

The idea of  peer review is critical when trying to determine if an article is scholarly.  Peer reviewed means that other experts in the field have looked at an article and determined that it’s worthy of publication.  Library databases help with this by identifying articles as academic , peer reviewed, or refereed (= another name for peer reviewed.) How is academic different than peer reviewed?  To begin with, academic articles are not peer reviewed; however, they are often written by experts in the field.  They may contain basic notes and a further reading list.  Scientific American  and Smithsonian magazine are two examples of sources that have academic, but  not peer reviewed articles.  If your directions say peer reviewed or refereed only, you can not use academic articles.

CAUTION:  There are two main problems with article database identification of scholarly journals:

  1. There are some journals that instructors disagree about (scholarly vs trade vs popular)
    1. These are generally trade/scholarly  journals in areas like marketing, teaching, criminal justice, etc.
    2. The specific instructor you are doing the research for wins when  there is a question about the scholarly nature of an article.
  2. All scholarly journals have non-scholarly articles in them.  These include:
    1. Letters
    2. Editorials & opinion pieces
    3. Book reviews

You can not use letters to the editor, editorials & opinion pieces or book reviews as scholarly articles,  even if they come from a peer reviewed/refereed journal.

NOTE:  Occasionally, even the experts get fooled.  You may remember the belief that the MMR vaccine can cause autism spectrum disorders, among other things.  This belief came from an article published in the British medical journal, The Lancet.  It turned out that the author fabricated most of the evidence.  The article has since been retracted, but it still pops on the web.  Moral of the story?  Be careful to have multiple sources backing up your beliefs when looking at controversial topics.

CURRENCY IN HISTORY SOURCES

You may be congratulating yourself on being able to use older sources for history topics.  Not so fast.   New resources are discovered all the time in all fields of history, including ancient history.  Can you use an older source?  Yes, if you have a new one that says similar things.  Can’t find a newer source?  Check with your professor and see if the older source is safe to use.

EXAMPLE:  a couple of years ago there was a newspaper article about the discovery of artifacts associated with the Roman Legions in Germany.  The newsworthy part of this was that the discoverers dated the finds to a time when the Roman Legions were not supposed to be in that part of Germany, and supposedly hadn’t been in the area for over  100 years.  The site is currently being excavated,   If the dating and the type of finds are what the original discoverers thought they were, then a lot of information about the Roman Empire in Germany will need to be rewritten.

The moral of the story is to always be sure that you have at least one current source in your bibliography/reference list.

 

EVALUATION CHECKLIST

You will need to use 2-3 questions from each category to evaluate most sources.  

There is one additional, very important,  thing you need to keep in mind when evaluating sources:

USE COMMON SENSE!

Relevance – will it be useful?

  • Is the information directly related to your topic?
    • If you’re writing a paper on how ballerinas dance on their toes, a resource about ballroom dancing isn’t acceptable.
  • Is there enough information?
    • With rare exceptions, such as statistics and facts (how long is the Nile River, how many provinces are there in Canada?), you need at least 2-3 paragraphs of information for a source to be useful.
  • Why do you think it will be useful?
    • How will you use the information?  As evidence to support your research question?  As an example?
  • Is it the right kind of information?
    • If you have to use scholarly sources, magazines, newspapers and general websites won’t work.

Credibility – is the information accurate?

  • Have you found similar information somewhere else?  
    • If you’ve found similar information in a different scholarly source, it’s probably accurate.
  • Does the author provide evidence and examples to support  his/her information
    • If an author says people prefer Coke to Pepsi, does s/he provide information from studies and research on the topic?
  • Is the author an expert on the subject?  
    • What is the source of his expertise – education?  experience?  
    • Can you tell what else the author has published?  Is it scholarly?   Popular, but in a respected magazine such as Archaeology?
  • Is there a way to contact the author?
    • Is there an email address?  An institutional affiliation (where they work)?
  • Are there references or a bibliography?
    • If you can use the sources and links to find other information, this is a good sign the information is accurate.
  • Is the information peer reviewed or edited?
    • Edited means at least one other person looked at the information and okayed it.  Peer review means 2-3 specialists in the field have okayed publication.  Editing is mostly found in popular sources.  Peer reviewed (or refereed) is found in scholarly sources.  Both indicate the source should be credible. 
  • For websites:
    • Is information about the website easy to find?
      • Can you find the author, sponsoring group, date created, etc.?  The better organized the site, the more likely it is a good source.
    • Is it a .edu, .gov or .org website? 
      • These sites tend to be more accurate (but not always – for example, students can often post information on many .edu sites.)
    • Are there a lot of ads?  Is the site trying to sell something?  
      • Some of these sites are good, but be sure to check for relevance, etc.

currency – how new is it?

  • Do you need current information?
    • If you’re writing on the terrorist group, ISIS,  your sources need to be very current.  If you’re writing on the pyramids in Egypt, older material may be helpful, but the majority of your sources should be from the last 10-15 years.
  • Can you tell when the  information was created?
    • This can be hard on a website.  A website without a date is often not a good source.  Remember that you often have to search for information on websites.

Objectivity – is there a bias?

  • Does the information clearly support only one side of an issue?
    • The Sierra Club website has a different view on logging than Pacific Lumber Company. 
  • Is the author trying to persuade you to come over to his/her view?
    • A website claiming the Beatles are the best band ever is not a good source of information on bands in the Sixties.
  • Does the information try to show both sides?
    • These sources are often a better, but be careful.  The author is picking and choosing his/her information.
    • There can be bias even where the author is trying to be objective.

     Most information is biased to some degree. 

The trick is to be aware of the bias and work with it.

  • Use the Sierra Club and Pacific Lumber Company websites as examples of opposing viewpoints. 
  • Use a website trying to persuade you to vote for a candidate as an example of campaign techniques in the U.S.

 

Leave a comment

Filed under Evaluation

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s