I present today, but I lost my voice! What do I do?
By Isaac D. Castillo (@isaac_outcomes)

If you present enough, you will encounter the situation I have today at #Eval14: You woke up this morning and have lost your voice. And of course, you present later in the day. How can you possibly recover in time to present?

Well, don’t worry. I am here to offer some advice that comes from decades of experience presenting in less than ideal situations.

First, do not panic. You can easily recover your voice for a 60 minute presentation. The key with recovering your voice comes in awakening your vocal cords. So, how do we do this?

Start by drinking hot beverages. Hot tea with lemon and honey is ideal, but any hot beverage will do. The hot liquids will help to loosen and awaken your vocal cords – getting them ready for use. However…..

Stay away from dairy products. As any speech coach will tell you, consuming dairy before a presentation is a bad thing even when you are feeling good. But if you have lost your voice, dairy will make things worse. Dairy products coat your vocal cords, making them more difficult to use. So, you can still have your coffee, but make sure you have it without milk or cream.

Next, keep eating. You may not feel great, but it is important to keep eating for two reasons. One reason is that you need your energy, and food is energy. If you are hungry, you will feel tired, and it will make your vocal cords feel worse. Another reason is that food will allow you to again get your vocal cords ready to use – although you aren’t talking, you are still moving your vocal cords when you swallow, which helps get them ready for use.

Finally – don’t use your vocal cords more than you have to. That means that you probably need to limit the number of conversations you have with people, especially right before your presentation. You are simply saving your vocal cords. Many people encourage gargling with mouthwash or water before you speak – but I think this is a horrible idea – it stresses your vocal cords at a time when you want to preserve them.

Right before you speak – drink some hot liquid, have a quick snack and speak aloud for 60 seconds to get your voice ready. Then start speaking. Make sure to have plenty of liquids at hand to drink while you talk – you will need them, but you will make it through! I promise.

Good luck and have a great presentation!

Who is Isaac? Before Isaac began working in the evaluation field, he was an All American college debater. He has taught public speaking, presentational speaking, and debate to high school and college students, as well as adults, for over 20 years.

Webinar dates: Tuesday, September 23, 2014 and Thursday, September 25, 2014 from 2:00 pm – 3:30 pm EST

Webinar feee: $75 for for AEA members and $100 for non-members

Register online: http://www.eval.org/eStudy

#omgMQP

I had the pleasure of spending Monday and Tuesday in Michael Quinn Patton’s Developmental Evaluation workshop. Due 10% to my bad vision and 90% to being starstruck, I sought out front-row seats:

Best seats in the house for the best speaker in the field. #omgMQP #eval13 pic.twitter.com/fQhXnPWBH3

— Ann K. Emery (@AnnKEmery) October 14, 2013

Along with many other nuggets of gold, MQP shared the Mountain of Accountability, a simple visualization demonstrating a Maslow’s hierarchy for organizations. (Start with the basics like auditing, personnel review, and outputs; then progress to typical program evaluation; then progress to developmental evaluation and strategic learning.) This visual was a fan favorite; the ipads and iphones were flying around as everyone tried to snap a picture. Anyone else think that MQP would be a great addition to the dataviz TIG?

A must see! “World debut” of @MQuinnP‘s Mountain of Accountability at #eval13 – like a Maslow’s hierarchy for orgs pic.twitter.com/WSZfk7fOmE — Ann K. Emery (@AnnKEmery) October 15, 2013

My biggest takeaway? Developmental evaluation is probably the future of evaluation, or at least the future of my evaluation career. Also, many evaluators wouldn’t call this approach “evaluation,” which means I’m not an evaluator, but an “evaluation facilitator.” Time to order new business cards!

#thumbsupviz

On Tuesday night I had Dataviz Drinks with Stephanie Evergreen, Tania Jarosewich, Andy Kirk, Johanna Morariu, Jon Schwabish, and Robert Simmon, along with a few more poor souls who had to listen to our endless enthuasiam about charts, fonts, and other things “worth staying up late for.” We’ve each been trying to reshape the dataviz community from one of frequent criticism to one of encouragement and peer learning (e.g., the Dataviz Hall of Fame.) A few beers later, the #thumbsupviz hashtag was born.

Stay tuned for our growing gallery of superb visualizations at thumbsupviz.com.

.@AnnKEmery plugs #thumbsupviz pic.twitter.com/qnLcdRmIbg

— Stephanie Evergreen (@evalu8r) October 17, 2013

omg Factor Analysis…

On Wednesday I attended a pre-conference workshop about factor analysis. I learned the approach in grad school a few years ago, have only used it twice, and wanted to brush up my skills. The instructor provided a wealth of resources:

#Eval13 workshop instructors have really gone above and beyond! Entire beautiful binder on factor analysis: pic.twitter.com/yJpuoB6wqn

— Ann K. Emery (@AnnKEmery) October 16, 2013

My biggest takeaway? Ouch. My brain was hurting. Leave the factor analysis to the experts because 99% of us are doing it wrong anyway. You don’t have to tell me twice!

Performance Management & Evaluation: Two Sides of the Same Coin

On Wednesday afternoon, I gave an Ignite presentation with my former supervisor and performance management expert, Isaac Castillo. Paired Ignites are rarely attempted, and I’m glad we took a risk. I had a lot of fun giving this talk. Stay tuned for future collaborations from Isaac and I!

Check out our slides and the recording of our presentation:

Excel Elbow Grease: How to Fool Excel into Making (Pretty Much) Any Chart You Want

On Thursday morning, I shared four strategies for making better evaluation charts: 1) adjusting default settings until your chart passes the Squint Test; 2) building two charts in one; 3) creating invisible bars; and 4) really really exploiting the default chart types, like using stacked bars to create a timeline or using a scatter plot to create a dot plot.

Here are the slides. The section about dot plots was pretty popular, so I recorded it later:

I thought the presentation went okay, but afterwards, an audience member came up to me and asked, “So if I wanted to make a different type of chart in Excel, like anything besides a typical bar chart, how would I do it? What could I make?” “That’s what I just spent the last 45 minutes showing you.” “No I mean, if I wanted to make one of these in Excel, could I do it?” “Weren’t you in the audience for the presentation I just did?” “Yes, that would be a cool presentation, you should show us how to make those charts in Excel.” Thanks for the great idea buddy, I’ll submit that idea to next year’s conference.

East-coast happy hour

For the second year in a row, the east-coast AEA affiliates got together for a joint happy hour on Thursday night. Good vibes and familiar faces.

The Washington Evaluators, Baltimore Area Evaluators, New York City Consortium of Evaluators, and the Eastern Evaluation Research Society

The Conference is Over, Now What? Professional Development for Novice Evaluators

On Friday afternoon I led a roundtable with tips for novice evaluators. The discussion was awesome, especially the great chats I had with people afterwards. I’m going to write a full post recapping that session. Stay tuned!

How to Climb the R Learning Curve Without Falling Off the Cliff: Advice from Novice, Intermediate, and Advanced R Users

On Saturday morning I had the pleasure of presenting with a former teammate, Tony Fujs, and my new teammate, Will Fenn. Tony dazzled the audience with strategies for automating reports and charts with just a few lines of R code, and Will shared tips to help novices avoid falling off the learning curve cliff. Check out their resources and tips in this handout.

Tony Fujs (left) and Will Fenn (right)

I thought the presentation went okay, but afterwards, an audience member commented, “It would be really cool if you got some evaluators together to show us what kinds of things are possible in R.” “Umm yep, that’s what we just did, Will and Tony showed how to automate reports and create data visualizations in R.” “Yep exactly, that would be a great panel, you could get several evaluators together and show how to automate reports and make data visualizations in R.” “Did you see the panel we just did?” “Yeah you should put a panel together like that.” Okay thanks, I’ll consider it…

Evaluation Blogging: Improve Your Practice, Share Your Expertise, and Strengthen Your Network

Dozens of evaluators have influenced and guided my blogging journey, and I was fortunate to co-present with three of them on Saturday: Susan Kistler, Chris Lysy, and Sheila Robinson. I first started blogging after watching Chris’ Ignite presentation at Eval11, Susan’s initial encouragement kept me going, and Sheila provides a sounding board for my new ideas.

Left to right: Susan Kistler, Chris Lysy, and Sheila B. Robinson

Can you tell we presented on Saturday morning?! Chris and I arrived early. I almost panicked, but instead Chris and I started laughing hysterically, and then a second person arrived. Close call!

By the time we started, we drew a good crowd of 30-40 bloggers and soon-to-be bloggers. Same time next year??

Evaluation Practice in the Early 21st Century

Where have we come from, and where are we headed? Evaluators have accomplished some amazing things, and the future is bright. Patrick Germain and Michelle Portlock, evaluation directors at nonprofit organizations, shared strategies for making evaluation happen when you are not in the room:

For me, the mark of a good presentation is when the evaluator shows vs. tells us something new. Kim Sabo Flores, Chad Green, Robert Shumer, David White, Javier Valdes, and Manolya Tanyu talked about incorporating youth voices into policymaking decisions. The best part: the panelists invited a youth participant to speak alongside them on the panel so that she could share her experiences firsthand.

They taught us about youth presence vs. participation, and then they showed us about youth presence vs. participation. Well done!

A dataviz panel shared a brief history of dataviz; strategies for displaying qualitative data; and ideas for using graphic recording:

One of many, many graphic recording examples shared by Jara Dean-Coffey

The Innovation Network team is pretty fond of graphic recording too, and Kat Athanasiades even recorded an entire advocacy evaluation panel. Thanks to Cindy Banyai for capturing this awesome video!

Graphic recording in action #eval13 https://t.co/8O1A7AWZhr

— Cindy Banyai (@RefocusInstitut) October 19, 2013

And just in case you’re not familiar with my plans for our field…

Wave goodbye to the Dusty Shelf Report!

Lookin’ good, Eval! See you next year in Denver!

Wear that nametag like a rockstar

Are you getting excited for the American Evaluation Association’s conference next week in Minneapolis?

This week I’m sharing conference tips for newbie evaluators and first-time conference attendees. While many of these ideas will be obvious for experienced evaluators, these ideas won’t be obvious for your coworkers and mentees who are attending one of their first conferences. Please consider sharing these tips, along with your own advice, with the newbie evaluators in your office.

Before the Conference

Here’s what newbie evaluators and first-time conference attendees should do before an evaluation conference:

• Assess your motivations for attending the conference.  Are you actively job-searching and want to conduct informational interviews during the conference? Are you considering starting your own consulting business and want to chat with independent consultants? Are you trying to meet others with similar interests so you can partner on future projects?

• Start using Twitter and LinkedIn to meet people and figure out who you want to meet face-to-face at the conference.

• Pack an extra casual outfit and an extra business outfit. Multi-day conferences tend to get more casual over time; that is, evaluators seem to wear suits the first day, then business casual for a few days, and nice jeans on the last day. Exception: If you’re presenting, save your favorite, nicest outfit for the day you’re presenting.

• Don’t forget your electronics. It’s easy to buy a replacement toothbrush from the hotel’s convenience store, but nearly impossible to find a replacement charger for your laptop or phone.

• Bring cash for taxis, tolls, tips, and parking meters. You’ll also need small bills ($1 and $5 bills) for group dinners. (Ever try to split a check between 20 evaluators with 20 different credit cards? It takes longer to pay the bill than eat dinner!) I usually pack $100 in small bills.

• Find out what you’ll be reimbursed for. For meals: Does your organization use a per diem system or a reimbursement system? For alcohol: Alcohol’s typically not covered. For hotel internet access: Unless you’re working during the conference to meet a client deadline, your organization probably won’t pay that extra $10/day for internet. Instead, scope out the free wifi hotspots in the convention center or conference hotel. For airfare: Is there a limit on your airfare? Will your company reimburse you for baggage fees? For ground transportation: Will your organization reimburse you for taxis to/from the airport, or just shuttle buses? For your timesheet: Will your organization provide a charge code for the entire conference, or just the first few days? It’s not uncommon to work “on your own” on the last day of a conference, especially if it falls on a Saturday.

• Find out which organizations will be represented at the conference. Even if you’re not job-hunting now, start brainstorming who you’ll want to work with in 5, 10, or 20 years. I like to skim through the back of the conference program.

• And if you’re presenting at the conference, follow the preparation checklist and slide design guidelines from the American Evaluation Association’s Potent Presentations Initiative. Evaluation conferences are different than research conferences (read more here) so adapt accordingly.

During the Conference

Here’s what newbie evaluators and first-time conference attendees should do during an evaluation conference:

• Iron your clothes and hang them in your closet as soon as you check-in to your hotel. Multi-day conferences are exhausting, and you’ll need all the sleep you can get over the next few days. I can’t tell you how many times I’ve overslept and ran to sessions wearing a wrinkled pair of pants…

• Keep your conference objectives in mind. Take charge and get what you came for! Networking won’t happen unless you initiate it.

• Take an evaluator to lunch (or breakfast, coffee, happy hour, dinner, etc…)
  

• Don’t work during the conference, even if it means working weekends beforehand to clear your schedule. If you’re presenting, don’t work on your slides during the conference (again, even if it means working weekends beforehand to finalize your presentation). Conference time is precious and is better spent learning and talking.

• That being said… Make time for self-care. Have quiet time in your hotel room, go to the gym, or take a walk outside.

• Skip a session or two for networking. My most valuable lessons and tips about evaluation have come from casual conversations with other evaluators, not from formal trainings.

• Trade business cards and jot down notes about each person to guarantee that you’ll remember them a few years from now (e.g. “Knows my friend Joe; lives in Baltimore but visits DC often; schedule coffee together.”)

• Tweet! Use the conference’s official hashtag (#eval12 for the next American Evaluation Association conference and #eers13 for the next Eastern Evaluation Research Society conference).

• Use Twitter to find #tweetups and happy hours.

• Attend a Topical Interest Group (TIG) business meeting at the American Evaluation Association conference. This is a great way to meet current and up-and-coming leaders in the field.

• Attend at least one session each on: 1) a brand new topic (to broaden your understanding of the field); 2) a beginner-level topic (to meet other newbies, and to feel more confident in what you already know); and 3) an advanced topic (to give yourself a reality check about what you don’t know).

• Ask questions during sessions. Don’t doubt their project, methods, or findings, or you’ll get a reputation for being an annoying audience member. I recommend straightforward clarification questions: “This is really interesting information, thank you. I’m new to this concept/project/approach. Could you give us a little more background about xyz?”

• Talk to presenters after their presentation. Don’t feel pressured to develop an elevator pitch or be an expert about their background or topic. Try this: “I learned a lot from your presentation, thank you. Where can I learn more about your work? Do you/your company have a website, blog, white papers, handouts, etc.? Do you ever visit DC? If so, would you like to have lunch the next time you’re in town?”

• Don’t arrive to sessions late or leave early. It’s bad manners and distracting to the presenters.

• Don’t worry about taking notes. Most handouts and slides will be available online right after the conference. My “notes” are my 140-character tweets.

• Wear that nametag like a rock star.

• And if you’re job-hunting… Print 100 copies of your resume and post them to the jobs board. Don’t share your resume with people you’re meeting for the first time unless they ask for it first.

After the Conference

Here’s what newbie evaluators and first-time conference attendees should do after an evaluation conference:

• Connect with everyone on Twitter and with a few people on LinkedIn.

• Send personalized, casual emails to 5-10 people you met. I’ve actually received emails with citations – “Hi Ann, I liked hearing about your work, it coincides with Smith (2010), who found a statistically significant difference between x and y. That is, until Miller (2011) employed a larger sample and found xyz….” Please don’t send anyone a literature review! Instead, write something like this: “Hi Ann, I liked hearing about your work. I’m going to apply xyz skill in my next project. Let me know if you’re ever in DC. I’d enjoy learning more over lunch sometime.”

• Schedule coffee, lunch, and happy hours with evaluators from your city.

• Share what you learned! Write on your personal blog, your organization’s blog, and/or the aea365 blog. Tweet. Lead a brown bag for your teammates. Bonus points: Explain what you learned to a non-evaluator (like a parent, roommate or significant other) without putting them to sleep. Translating interesting ideas from evaluationese into lay language is a skill that improves with time.

Have additional tips and tricks to share with newbie evaluators and first-time conference attendees? Please share your ideas below.

Thanks, Ann Emery

Additional Resources:

• American Evaluation Association members can watch Susan Kistler’s coffee break webinar, Ten Tips for Making the Most of the Annual Conference.
• Tips for First Time Conference Attendees by Susan Kistler on aea365
• Making the Most of the AEA Annual Conference by Bonnie Stabile on aea365

“This year, the American Evaluation Association is loaded with a ton of great stuff… I’m also really excited to see the closing session on Saturday, because AEA President Rodney Hopson is pulling together something amazing.

The theme of the conference this year is Evaluation in Complex Ecologies: Relationships, Responsibilities, Relevance. What in the heck does that mean, right? Well, this is your chance to help define it. Take a picture, wrap it in 25 words, and send it to AEA2012closingphotos@gmail.com before noon CT on Thursday, October 25. The best photos will be chosen and featured in the closing session on Saturday October 27, 2012 at 4:30 pm. Dudes, there will be prizes.

If it isn’t obvious to you already, what’s so so cool about this project is that it is an attempt to let people get visual with concepts that are pretty darn abstract. We, serious and academic researchers and evaluators, often shy away from exploring a bit of creativity. This is an officially sanctioned license to go play (even for those of you who aren’t headed to the conference this year – you can still participate).”

Although I was tempted to submit something like this, here’s my submission:

Internal evaluator seeks stakeholder buy-in and learning culture. Willing to wear hairnets, plastic gloves, and serve tortilla chips to 500+ children during organization’s Thanksgiving dinner.

When I checked the schedule and realized the sessions start at 7am:

When I’m already flooded with work before the conference:

When I still have to finish a few presentations before the conference:

When I thought, hey, it’s not so bad, because this is how I felt at my first evaluation conference:

When I remembered there would be be plenty of happy hours at the conference:

And when I remembered I’d get to see all my favorite evaluators in one place:

Enjoy! Ann Emery

——————————–

Isaac Castillo

“One of the most difficult things for many presenters to master is volume control of their voice.   In most situations, we are used to talking in a normal conversational tone and volume.  But normal volume usually doesn’t work for a presentation – and it certainly doesn’t work if you are presenting without a microphone.   On the other hand, you don’t want to be screaming at your audience, particularly in a small space.

So how do you know if your volume is correct?  Here are some tips to keep in mind.

If at all possible, get to your presentation space early and ask someone to help check your volume.  This could be a friend or colleague, the person responsible for audio/visual needs, or even that eager person that shows up first.  Keep in mind that what sounds loud enough at the beginning of your presentation in an empty room will NOT be loud enough when dozens (or hundreds) of people fill the room.

But sometimes you won’t have the chance to check your volume in a room before you start.  In these cases, here are some things you can do to make sure everyone can hear you.

First, you can simply ask your audience if you are loud enough.  There is nothing wrong with this, and really is one of the best ways to check your volume.  The key is to make sure you get feedback from the back of the room – those in the front row will always be able to hear you, you want to make sure the same is true for those in the last row.

Second, you should look for signs that members of the audience are having difficulty hearing you.  Looks of confusion, people leaning forward, and tilted heads are all good signs that you are not being loud enough.  And again, pay attention to those in the back row.

Tips for Beginners:   Talk slightly louder than you think you should be talking.  If you feel like you are talking in a normal voice, you are not loud enough.  You should really feel like you are talking slightly louder than necessary – that will be your correct volume for a filled room.

Tips for Veterans:  Work out your diaphragm!  Your diaphragm is where you get your vocal volume.   Do this exercise to increase your capacity to talk louder:  lie on your back and place a book (nothing too heavy) on your stomach directly below your ribcage.   Take a deep breath in and lift the book by inflating your diaphragm.  Breathe out and let the book fall with your stomach.  Practice this for five minutes a day – increasing the weight of the books.  You should eventually be able to lift two telephone books with no effort.

Tips for Experts:   Use different volume levels during your presentation.   Telling stories or communicating content at different volume levels can add a different level of emphasis to your presentation.  I like to use a softer voice when talking about difficult work or when telling emotional stories.  Louder voices are good for exciting moments.  Varying your volume intentionally within your presentation can help make a good presentation great.”

–Isaac Castillo

Karen Anderson is my favorite “newbie” evaluator. Karen completed a master’s degree in social work a couple years ago. She’s currently an evaluator at a nonprofit in Atlanta and she’s the Diversity Programs Intern for the American Evaluation Association. In all her “spare” time, she’s doing pro-bono evaluation for the State of Black Gay America Summit organizers. And she’s a blogger!

You can read Karen’s LinkedIn profile here, and you can read her blog, On Top of the Box Evaluation, by clicking here.

I hope you enjoy Karen’s guest post.

— Ann Emery

———-

When I think about the “newbie” evaluator or the not so new professional to the evaluation field and the necessary knowledge base and skills needed to not only survive, but to thrive, I reflect upon Jean King’s Essential Competencies for Program Evaluators.

King’s Essential Competencies for Program Evaluators include:

1. Professional Practice: These are the fundamental norms and values of evaluation practice, which include working ethically, applying evaluation standards, and considering the public welfare, which is further explained in the AEA Guiding Principles for Evaluation under Responsibilities for General and Public Welfare.
2. Situational Analysis: The unique interests, issues, and contextual circumstances of evaluation.
3. Reflective Practice: One’s own evaluation expertise and need for growth, which includes knowing self, reflecting on your practice, pursuing professional development, and building professional relationships.
4. Interpersonal Competence: Do you have the people skills for evaluation practice? This includes negotiation skills, conflict resolution, cross cultural competence, and facilitating constructive interpersonal interactions.
5. Project Management: King describes this as the “nut and bolts” of evaluation work” in the presentation above. This includes presenting work in a timely manner , budgeting, responding to RFPs, use of technology, and supervising and training others.
6. Systematic Inquiry: The technical aspects of evaluation. What’s your knowledge base? Do you know qualitative, quantitative, and/or mixed methods? Developing program theory, evaluation design and evaluation questions are also major components of this competency area.

So in terms of the Essential Competencies for Program Evaluators, how are you doing? I wish I had some type of rating scale to help me to see how far I’ve come along. I’d have to say on the job training, webinars, and seeking out evaluation trainings, no matter how brief, like the online American Evaluation Association’s Coffee Break Demonstration Series has helped me to come a long way since my grad school days (2010).

Newbies: What are some “essentials” that you think are missing from above that relate to your evaluation practice growth and development?

Not so newbies: What steps do you take to sharpen your evaluation skills and to increase your knowledge base?

— Karen Anderson

Even though the conference was a couple weeks ago, I’m still thinking about one of Rodney Hopson’s comments. He mentioned that sometimes he wonders whether evaluators/the evaluation are actually hurting the program rather than helping it.

I’ve certainly had similar experiences. Mostly, I’ve seen program staff get so excited about data that they want to collect more, and more, and then even more data. You can read about one of my experiences here.

This is a great idea at first. What’s the harm? More data is better, right?

But… a few months down the road, the program staff and I are swimming in more data than we can handle. And, we often have more data than we really need. After all, my goal as a utilization-focused evaluator is to collect information that will directly influence decisions about the program or the participants. Simple, quick, streamlined data can be more useful than complex, time-consuming data.

Have other evaluators felt like this? Have you ever questioned whether your involvement is hurting rather than helping?

I hope you enjoy reading Mika Yoder Yamashita’s explanation about the best presentation she’s seen. Mika’s my evaluation friend from the Washington Evaluators, and the best presentation she’s seen was a systems dynamics presentation at the Stevens Institute.

— Ann Emery

————————————–

“The best presentation I saw was a presentation by a professor at Stevens Institute, and it was a quick walk through of systems dynamics (a part of systems thinking).  It was 20 minutes (and the presenter mentioned at the beginning that he would be talking for 20 minutes).   There are a couple of factors that I think made me to think this presentation was great.

First, there was a kind of book reading assignment.  The presentation happened as a part of a book club meeting.

Second, the book was difficult to me.  I did not read deep into systems dynamics before beyond Sociology 101 class, so it took a lot of my brain to go through the book.

When the presenter made the overall argument in the theory into a  concise diagram, I thought “we are seeing the same thing.” Yes, the presentation lost some details, and because it was simplified, I am sure the message was open to interpretation.

The purpose of the presentation was not to convey the same message. It was a warm up or quick introduction to a discussion.

Of course,  we could see the diagrams projected on the wall, and the presenter used probably 10 slides or less.   I do not remember if there was a handout, but since the presenter walked us through the diagrams, no need to read a handout.”

— Mika Yoder Yamashita