AI: What makes a system expert?

Artificial Intelligence (AI) is and has been on people’s minds for a long time. Advertising agencies and marketing experts talk about:

  • What does Artificial Intelligence mean for marketing agencies?
  • How Artificial Intelligence Is Revolutionizing the Digital Marketing…

Such titles promise much more than most of the blog or webpage entries deliver.

One of the criticisms of AI is that such systems are unable to ace an eighth-grade science exam. The main reason being that current AI systems:

“…[cannot go] beyond surface text to a deeper understanding of the meaning underlying each question, then use reasoning to find the appropriate answer.” (p. 63)

Schoenick, Carissa, Clark, Peter, Tafjord, Oyvind, Turney, Peter, and Etzioni, Oren. (September 2017). Moving beyond the Turing Test with the Allen AI Science Challenge. Commun. ACM 60(9), p. 60-64. DOI: https://doi.org/10.1145/3122814
Check out the video at the bottom of this post !

Read the rest of this blog entry to:

  • define what an expert system is;
  • show why Pinterest’s updates are based on an imperfect AI system;
  • illustrate the challenges of using AI to augment marketing;
  • watch an interesting video about AI and learning science further below; and
  • ask you for your feedback, input and opinions – join the discussion.

This entry is part of our series of posts on AI. To stay tuned and get the latest updates, including on AI and marketing, sign up for our newsletter.

This project is part of our White Paper project for the Competence Circle Technology, Innovation and Management #ccTIM from the German Marketing Association (Deutscher Marketing Verband).

This post continues our discussion entitled, What is marketing automation?

1. Definition of an expert system

In the 1980s, we were all interested in Decision Support System(s) (DSS) and expert systems. The use of AI garnered a lot of interest from the business press.

Using AI became easier, at least in theory, thanks to the rapidly decreasing costs of calculating or doing the arithmetic for ever larger data sets. This made it feasible to use many mathematical operations to gain insights into user and customer behaviour.

At the same time, AI systems represented the risk of amplifying implicit bias contained in the data sets they were trained on. In turn, some systems can make wrongful inferences or judgments about users. Below we attempt to define what an expert system is.

Defining an expert system

An expert system uses specialised knowledge and expertise from a human expert in a particular problem area and converts it into software code. With the help of such code, the expert system can emulate the decision-making ability of a human expert. It allows the system to perform at a level of competence that is better than that of non-expert humans.

Expert systems are part of a general category of computer applications known as artificial intelligence.
Expert systems can be used to diagnose patients, to put together a system that identifies fake news, and so on. Difficulties can arise when interpreting results produced by “black box” systems whose workings are often hard to analyse.

Edward Feigenbaum is seen as the father of expert systems.

See also definition by Encyclopaedia Britannica.

Of course, in cases where decisions can be clearly defined with one or even many algorithms (i.e. mathematical operations), we expect expert systems, and thus computers, to take over most of the tasks currently done by humans.

For an expert system to work well, two things are paramount:

  1. its rules and algorithms need to work properly, and
  2. the rules and decisions made need to be the right ones.

Hence, expert systems are often downgraded to represent expert support systems, which support humans in making better decisions. We define expert support systems below.

Defining an expert support system

An expert support system helps people solve problems. Like an expert system it allows the system to perform at a level of competence that is better than that of non-expert humans.

For instance, with Legalos, the user of the expert support system enters relevant information. The expert support system then uses this information and generates a template, for example a contract between a company and its cloud services provider. Here, the expert support system can provide the entrepreneur with several types of standard contracts very quickly. In turn, this helps keep a company’s legal costs down.

Another simple online expert support system is provided by Germany’s Federal Ministry of Justice and Consumer Protection. The service asks the user to enter some information pertaining to data processing and privacy measures. Based on this input it then generates a transparent data privacy policy as required by Article 12 of the General Data Protection Regulation (GDPR). This can then be slightly modified to fit the company’s particular circumstances.

See also: Luconi, Fred L., Malone, Thomas, W. & Scott Morten, Michael, S. (December 1984). Expert systems and expert support systems: The next challenge for management. Boston: MIT working paper #122, Slong wp #1630-85. Retrieved 2018-06-12 from http://dspace.mit.edu/bitstream/handle/1721.1/47478/expertsystemsexp00luco.pdf

In general, an expert system must acquire knowledge from experts. Such insights are then applied to a large set of probability-based rules to make a decision.

By contrast, an expert support system still requires the human user to weigh some of the factors and then arrive at a decision.

2. Pinterest updates – more noise

Many companies use such technology. For instance, Pinterest and Instagram use similar AI to figure out what Pins you should check out on Pinterest or which Instagramers you should follow. Twitter operates the same way, and so does Facebook (see your newsfeed) or LinkedIn (whom you should connect with).

Recently, I got just such an update (see image below), suggesting that I go and check out 18 pins I should be interested in, based on my board #MCLago.

How on earth did Pinterest's "expert system" decide that these pins are relevant to my #MCLago board?
How on earth did Pinterest’s “expert system” decide that these pins are relevant to my #MCLago board?

3. When expert systems fail to augment marketing

As you can see in the image above, whatever criteria Pinterest used to determine what pins might be of interest to me, ‘common sense’ was not programmed into this decision-making process. How it concluded that I wanted to meet single men is a mystery to me.

Why I should care about Lipitor – a prescription drug – is unclear. Yes, I do post medical stuff, but primarily about minimally-invasive endoluminal or endolumenal surgery, because of my work with Lumendi Ltd.

On the upper left in the above screenshot you can see some people in a photo. The program concluded this from one of my recent pins. I had recently posted something – with video – about a Syrian refugee (the picture shows the trainee with her co-workers and bosses). So the thought was I would like another one. Well, here a deeper understanding of the meaning underlying the item I pinned would have allowed Pinterest’s expert system to find a picture in a similar realm.

Instead, it inferred that I would be interested in “Who’s In and Who’s Out for the Next Season of Nashville“. Seems a little ridiculous.

Basically, an expert system needs to be able to do more than do simple math. Moreover, predictions are not enough to automate the decision-making process or task with the help of AI (see Agrawal, Gans & Goldfarb, Spring 2017). Below, we list the six key things an expert system must be able to handle to get AI to deliver the most value.

Agrawal, Aja, Gans, Joshua S. & Goldfarb, Avi (Spring 2017). What to expect from artificial intelligence. MIT Sloan Management Review, 58(3), pp. 23-26. Retrieved 2018-06-12 from https://sloanreview.mit.edu/x/58311

Expert system for automation: Performing these 6 actions is a must

An expert system not only executes tasks efficiently, but more importantly, gets a few things right, such as:

1. Data analysis: What kind of photos or status updates does this individual post?

2. Prediction: What action would the recipient take and / or would this potentially be of interest to the customer?

3. Judgment: Yes, this status update / photo is of interest to the user / customer.

4. Action: Include photos of interest and mail out newsletter to subscriber, user or customer.

5. Key Performance Indicator (KPI): The recipient has clicked on several of those 18 suggested pins. This expert system did better than average.

6. Quality of service: The pins the client clicked on provided content that represents added value for this user.

Unless the expert system we use can do the above, marketing activities are more likely hampered than augmented.

4. Ultimate test: Does this content answer the question I am asking?

As pointed out above, whether the user clicked on several suggested pins is one possible KPI. For instance, I clicked on more pins than could be expected. Nevertheless, ultimately it is not the clicks on pins recommended to me by Pinterest that matter. Instead, the ultimate criteria for a user is whether those pins provide information that represents added value.

In my case, that did not apply. To illustrate, I checked out the pin about 10 KPIs in marketing, which brought me to a blog entry (see image below).

When an expert system cannot deliver quality: Pinterest recommends pins that mean little or nothing to me.
When an expert system cannot deliver quality: Pinterest recommends pins that mean little or nothing to me.

As the above shows, somebody is spreading her opinions regarding KPIs. We all know that the life cycle of a client is important, but if you are running a start-up, this could be of lesser importance than getting new clients who can help you pay the rent.

Strategising your sales revenue approach is interesting, but not something that everybody needs to do. Treating your clients respectfully and providing a service that they feel is worth the money they paid you most certainly helps. When it comes to revenues, that applies regardless if you track it with a spreadsheet or do it on a piece of paper.

5. What is your opinion?

The verdict is simple. The expert system that Pinterest uses to serve me weekly or more with an email of suggested pins does not do a good job. The recommendations it makes indicate that the AI system lacks a deeper understanding of the meaning underlying each pin I uploaded. In turn, it cannot source pins that might interest me.

But do not be fooled, neither Twitter nor Instagram do better with these things. Developing a well-functioning expert system takes a lot of work and testing.

However, the fact that expert systems do make errors was already pointed out by researchers in the 1990s:

Williams, Joseph (1990). When expert systems are wrong. In Proceedings of the 1990 ACM SIGBDP conference on Trends and directions in expert systems (SIGBDP ’90), p 661-690. DOI: https://doi.org/10.1145/97709.97761

On reviewing the challenges and benefits of expert systems and neural networks, things do not appear to have become easier in 2014, even though the benefits can be substantial (e.g., https://link.springer.com/article/10.1007/s10916-014-0110-5).

What I would love to know is what you think about these issues in 2018 (#ccTIM will continue updating you on this subject):

  • Do you think AI (artificial intelligence) will revolutionise marketing? Please explain why or why not.
  • Do you have examples of great expert systems, for instance in marketing, management or production?

The author declares that he had no conflict of interest with respect to the content, authorship or publication of this blog entry (i.e. to the best of my knowledge, I got neither a freebie from any of the aforementioned companies, nor are they our clients).

Check out this video, worth watching – see quote at the beginning for reference to research paper that is source for video below.

Urs E. Gattiker

Professor Urs E. Gattiker - DrKPI is corporate Europe's leading social media metrics expert (see his books). He continues to work with start-ups. Urs is CEO of CyTRAP Labs GmbH and President of the Marketing Club Lago, a member of the German Marketing Association (DMV).

11 thoughts on “AI: What makes a system expert?

  • 14. June 2018 at 10:01
    Permalink

    Dear Urs

    Subject errors I cannot tell you about regarding your definition of an

      expert system or an
      expert support system

    However, I thought I look at this from a “common sense” perspective and the average consumer

    I read this blog entry and I find it gives me a critical assessment. But it reflects the spirit of time, whereby we have gotten to the stage were we try to critically re-assess if our euphoria regarding, e.g., bots is really justified.

    Nevertheless, maybe you are a bit too critical

    Hope this is useful.
    Cathrin

    Reply
    • 14. June 2018 at 10:05
      Permalink

      Dear Cathrin

      Thanks so much for taking the time answering.

      I am not sure if I would call my assessment critical.
      Maybe it is just realistic. In other words, in the 1980s we talked about the office becoming paperless. Last time I looked and regardless which of our clients’ offices I visit, plenty of paper to be seen.

      I believe similarly to the paperless office, AI is a great idea but as one of the people who made up the term “artificial intelligence” pointed out, it takes a lot of effort to get it right.

      Prof. John McCarthy (English)
      Prof. John McCarthy – German text

      We have maybe come a long way, but the journey seems to just have started.
      Greetings
      Urs

      Reply
  • 14. June 2018 at 10:03
    Permalink

    Dear Urs

    I thought I have to reply to your Pinterest example.

    Your verdict is clear, it ain’t working for you.

    But could it be a reflection of your “wild” or non-systematic post as well as the fact that you may not post as a person but as MCLago that the suggestions provided are not that great?
    I have a Pinterest Account as well. I have yet to upload a picture. Instead I pin things/pictures to my wall that are of interest to me. The suggestions that I get from the system are beginning to be quite a good fit and of interest to me… However, I must admit, I use Pinterest basically for ONE theme only (i.e. what I pin to my wall is from one area of interest only).

    With Facebook, I am often surpirsed how exact or insightful the suggestions made are regarding new friends. Even some people that are not part of my contacts on my mobile are part of those people the system recommends me to connect with or befriend via Facebook.

    Naturally, we have to feed the system with information first before it can provide us with insightful suggestions.
    I know of a case where the person has 2 contacts. The individual rarely if ever visits the platform. In turn, the suggestions FAcebook makes regard whom the person should connect with make little sense.

    A clear case where Facebook does not have enough data points to provide insightful recommendations of people my mom should or could likely want to connect.

    Hope this is useful.
    Cathrin

    Reply
    • 14. June 2018 at 10:26
      Permalink

      Dear Cathrin

      Thanks for this second reply.
      I agree with your assessment (e.g., Facebook) that the more data points the social network has through my postings, likes, comments, etc., the better are the suggestions regarding whom I should connect with.

      You also write:

        But could it be a reflection of your “wild” or non-systematic post as well as the fact that you may not post as a person but as MCLago that the suggestions provided are not that great?

      I actually post as a person on Pinterest
      Maybe what makes things difficult is that I have several boards?
      Each has a somewhat different focus but most deal with Strategy, Marketing, Cost-Benefit Analysis, Metrics/KPIs.
      But with the #MCLago my focus is on the Marketing Club Lago – related events and matters (e.g., German Marketing Association)

      The example still suggests that Pinterest has a hard time to fine-tune its expert system so that it can interpret the pictures I post in the larger context of a board such as #MCLago.
      At this stage, I believe the task is too difficult for Pinterest’s expert system and a human expert would be required to make the final assessment. But of course, this would be too expensive, thus Pinterest continues using a machine to do the selection and the result is dismal.

      But it also confirms what I stated in the blog entry that simple and non-complex tasks can easily be managed by a program that uses algorithms, in order to make plenty of decision very quickly. An example might be:

      1. are there people in the picture? If yes go to:
      2. if there are people in the picture, are these men? If yes,
      3. are there also women in the picture? If yes,
      4. are there also kids (< under 14 years of age) in the image?

      As this indicates, tons of decisions that need to be written down properly to reduce the possible errors that could happen by the system making the wrong decisions (or executing an algorithm incorrectly).

      I am still waiting for Twitter doing better in what people it suggests I follow, for instance. Currently I am getting a potpourri of suggestions.
      Thanks for sharing Cathrin and I will take your input into consideration for writing our first draft of the #ccTIM paper about artificial intelligence and marketing.

      Merci
      Urs

      Reply
  • 23. June 2018 at 7:57
    Permalink

    Thank you, Urs for your inspiring blog post.

    I cannot judge the value perception of your arguments, because marketing is not my key business.

    My expertise is in learning and education. Nevertheless, AI is an exciting field of interest for me.
    Therefore, I like to share here two blogpost from the Chatbots magazine.

    The first blogpost is the Chatbot Report 2018: Global Trends and Analysis .
    with an overview to the development of Chatbots since 2010 and with some interesting statistics. I agree with the statement:

      “One of the basic reasons of chatbots expansion is so called “being tired of apps”. Consumers are annoyed by the need to install special applications to their mobile devices.”

    The second post is about learning: 6 Ways Artificial Intelligence and Chatbots Are Changing Education

    Feedback is important in marketing and in education. Hence, I am really curious to see how this technology will further develop considering the feedback loop, i.e. will machine learning accelerate this development or does it promise more than it can deliver?

    I will continue to follow with great interest the further development of chatbots thanks to AI. Will chatbots get better but as well, will learning with the help of chatbots, expert systems or robots help in improving eduction programs and health care.

    Best regards
    Yvonne

    Reply
    • 23. June 2018 at 10:48
      Permalink

      Dear Yvonne

      Thanks so much for taking the time to provide us with these links. I subscribed to the chatbot’s newsletter via e-mail.
      And before I reply, let us not forget your very interesting entry about robots in health care services

      The biggest challenge seems to be the effort it takes to get these chatbots to work with more difficult tasks.

      IBM just confirmed that it took 6 years to program a Watson derivative to be able to debate two humans about the space program and if it should be supported by public funds and why.

      German Article: KI und Menschen können erstmals streiten
      English Article: IBM computer holds its own in debating contest

      IBM Video about the event: IBM Project Debater

      Especially the IBM Project Debater shows that machine learning is still in its infancy and it takes years to do something complex ….

      What the teaming up of Amazon with Marriott will accomplish, when each rooms will offer guests Alexa for services will be an interesting case.

      Alexa and Marriott: Amazon teams with Marriott to put Alexa in hotels.

      Apparently, these devices will do many things from playing music to controlling the lights to ordering room service. That is nice but if this simplifies my life as a guest at one of their properties remains to be seen. I will gladly test it out….

      Thanks for sharing and have a nice weekend.
      urs

      Reply
  • 23. June 2018 at 12:21
    Permalink

    Thank you Urs
    The comment and your references regarding IBM, Alexa and Marriott is particularly interesting

    Have a nice weekend too.
    Yvonne

    Reply
    • 23. June 2018 at 17:17
      Permalink

      Dear Yvonne

      Glad to hear back from you. Chat bots is a fast growing business.

      Besides SEO there is Personal Assistant Search Optimimization or PASO that is becoming ever more popular for voice marketing. Alexa hopes to learn your preferences, but PASO tries to reach the customer before he even starts looking for a brand or gives Alexa shopping instructions for chocolate or toilet paper.

      In turn, he might choose our brand thanks to PASO (AI … coming slowly but ever more).

      Thanks for sharing
      urs

      Reply
  • 26. June 2018 at 0:34
    Permalink

    Dear Urs,

    After reading your post I wondered if there is ONE AI only? I guess not, as AI is developping, it is further develping and it changes on the way or is further refined.

    This is true for marketing as well. Over the year it has changed

    Nevertheless, I believe AI will be an important element in the marketing world. On purpose, I refrain from using the word tool because AI will go beyond just being a tool.

    Marketeers will be able to brief an AI system to get best practices for business challenges, for effective communication etc. Furthermore, marketers will be able to identify patterns. Criteria used need to be modified.

    This systematic approach will give the impression that it helps reduce uncertainty. Moreover, the quality of AI will determine our satisfaction with its outcomes and how accurate the forecasted outcomes will materialise.

    However if everyone in the same industry is using the same AI, everything will be streamlined. Some questions that remain with such a scenario are such as:

    1. If we use similar AI techniques and approaches, how will our products remain innovative?
    2. How good does AI work for niche markets or products. Put differently, will it be mainly used for mass market products but not customised ones?

    The near future should give us some indications of how things develop if not answers.

    Regarding your question if AI will be a revolution?

    My answer is yes. Besides AI changing the way we construct machines or appliances, I think we as consumers or humans will more likely accept machines or artificial intelligence as part of our daily lives.

    This will mean we need to change in marketing, consider this change in attitude toward AI and adjust our marketing accordingly.

    What can be trusted by humans?
    Maybe personal relationships will be even more special and important than today for humans to feel alive and grounded.

    Looking forward to our future, also as humans 🙂
    Alexander

    Reply
    • 27. June 2018 at 15:49
      Permalink

      Der Alexander
      I am splitting my reply due to length.

      First, let me thank you for taking the time and answering to this blog entry.

      Reply 1
      I totally agree with you, there are many shades of grey and that applies for AI as well.

      Nevertheless, I felt we had to define the term in order that we compare apples with apples and not apples with oranges. Long time again Thomas S. Kuhn wrote The Structure of Scientific Revolutions (first edition was published in 1962, 50th anniversary edition in 2012) pointed out that well established scientific disciplines share a vocabulary, values and attitudes.

      In turn, the shared vocabulary makes it much easier to know that you talk about the same type of apples when discussing orchards and trees.

      I think when it comes to artificial intelligence we are still far away from sharing a vocabulary.
      Thanks
      Urs

      Reply
    • 27. June 2018 at 15:55
      Permalink

      Alexander

      Reply 2

      Recently I cam across an article in a newspaper about 3-D printers and dreams. Amongst other interesting things it talked about that besides the wiring and circuit diagrams we also need to consider humanistic parameters since these could change.

      For example, the augmented or “improved” human that can run faster and jump higher thanks to neurotransmitters and sensors, degenerates towards the extended workbench of technology firms.

      You write:

        “However if everyone in the same industry is using the same AI, everything will be streamlined. Some questions that remain with such a scenario are such as:
        1. If we use similar AI techniques and approaches, how will our products remain innovative?
        2. How good does AI work for niche markets or products. Put differently, will it be mainly used for mass market products but not customised ones?
        The near future should give us some indications of how things develop if not answers.”

      These are important questions. Nevertheless, we can use similar AI techniques but our products will differ. When the iPhone was released all companies had access to the technology but surely, we can agree the iPhone was different then all the Nokia versions and so forth.

      Similarly, the IBM PCJr was different than the Apple IIe but it made the personal computer respectable in the corporate offices across the USA and elsewhere. IBM became somewhat dominant for a while but 1984 Apple fought back as the lovable underdog:

      For your first question, my answer is yes… what we do with the tools we have will result in different outcomes and products… as long as we let human ingenuity prevail.

      Your second question regarding niche markets and products is a tough one as well. As I explained to Yvonne above in my reply:

        IBM just confirmed that it took 6 years to program a Watson derivative to be able to debate two humans about the space program and if it should be supported by public funds and why.

      Today it seems it is complex. The simpler the algorithms we use – see https://DrKPI.com where we benchmark content, marketing campaigns, and voice marketing – the faster you can make AI work for you relatively inexpensively. If you take on IBM’s challenge to create a system that can debate experts, a few years of work be a ton of experts is needed.

      But of course, we do not make the mistake to think humans are not getting better at this 🙂
      So my answer to your second question is, it can work depending on if you want to do it today or tomorrow and how many resources you can or are willing to put in….

      Let us do it.
      Greetings
      urs

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *