Deep Learning for Cognitive Computing, Theory

(Course code: TIES4910) 5 ECTS,      Autumn Semester

Instructor:

Vagan Terziyan               Email: vagan.terziyan@jyu.fi

 

 (Find the course and register in SISU system)

 

 

 

 

 

 

Attention: 1-st Lecture: Thursday, 12 September 2024. Time: 12:15 - 14:00. Place: Ag B122.1 (Alfa). (Check also in Moodle)

 

===========================================================

NOTICE:

 

If any content linked to by pages of this site has the prefix:

 

http://www.mit.jyu.fi/ai/ ...

 

… or the prefix:

 

 http://www.cs.jyu.fi/ai/ ...

 

… and is not accessible, then use the new prefix:

 

https://ai.it.jyu.fi/ ...

 

… to access.

 

Sorry for the temporary inconvenience due to an unexpected server change.

===========================================================

 

COURSE SCHEDULE (FALL-2024)

A screenshot of a calendar

Description automatically generated

 

 

THIS COURSE IS INDEPENDENT PART OF OUR ARTIFICIAL INTELLIGENCE COURSES’ PACKAGE; IT DOES NOT REQUIRE ANY PREQUESITIES AND CAN BE STUDIED REMOTELY

Personal quota from the course instructor:

 Everything will be Artificial Intelligence soon, and, instead of being worried about it and restricting it (which is a bad idea), come to study, develop, use, and promote it further!

Image preview

 

 

Artificial Intelligence (AI) is a major driver of economic growth and social progress, if industry, civil society, government, and the public work together to support development of the technology, with thoughtful attention to its potential and to managing its risks, as it mentioned in the Report on the future of AI from the White House Office of Science and Technology Policy. One of the most emerging trends within AI nowadays and in observable future is Cognitive Computing and its enabler – Deep Learning.

The pair of courses: this one – TIES4910-Deep Learning for Cognitive Computing, Theory and its extension – TIES4911-Deep Learning for Cognitive Computing for Developers (5+5=10 credits, delivered in English) is an evolution of the course ITKA-352: “Introduction to Watson Technologies”, which aims to provide more systematic, structured (broader, deeper and multidisciplinary) view to this popular domain. The major objectives of the course are as follows:

·        To describe challenges and opportunities within the emerging Cognitive Computing domain and professions around it;

·        To summarize role and relationships of Cognitive Computing within the network of closely related scientific domains, professional fields and courses of the faculty (e.g., Artificial Intelligence, Semantic and Agent Technologies, Big Data Analytics, Semantic Web and Linked Data; Cloud Computing, Internet of Things, etc.);

·        To introduce the major providers of cognitive computing services (Intelligence-as-a-Service) on the market (e.g., IBM Watson, Google DeepMind, Microsoft Cognitive Services, etc.) and show demos of their services (e.g., text, speech, image, sentiment, etc., processing, analysis, recognition, diagnostics, prediction, etc.);

·        To give introduction on major theories, methods and algorithms used within cognitive computing services with particular focus on Deep Learning technology;

·        To provide “friendly” (with reasonable amount of mathematics) introduction to Deep Learning (including variations of deep Neural Networks and approaches to train them);

·        To provide different views to this knowledge suitable to people with different backgrounds and study objectives (ordinary user, advanced user, software engineer, domain professional, data scientist, cognitive analyst, mathematician, etc.);

·        To discuss scientific challenges and open issues within the domain as well as to share with the students information on relevant ongoing projects in the Faculty;

·        To train within teams to use available cognitive services via GUIs or APIs for inventing new interesting use cases and designing own applications;

·        For advanced students there will be a possibility to contribute (enhance, optimize, etc.) known algorithms or the related science behind them.

We believe that knowledge on Cognitive Computing at least at the level of an advanced user of it would be an excellent added value within the portfolio of every professional (from very humanitarian to very technical one).

Interesting to notice, that the major companies provide support to deep machine learning architectures for cognitive computing with an appropriate hardware. For example, Intel recently announced a new chip called Loihi, which is neuromorphic and self-learning chip capable of representing 130 000 neurons and 130 million synapses. Unlike convolutional neural network (CNN) and other deep learning processors the Loihi chip uses an asynchronous spiking model to mimic neuron and synapse behavior in a much closer analog to animal brain behavior.

Important challenges around the course topics also include Cybersecurity-related aspects of the Cognitive Computing, Deep Learning and Collective Intelligence. Emerging Cognitive Computing services attract huge amounts of users worldwide. Very recently the new vulnerabilities of Cognitive Computing and of its enabler Deep Learning have been discovered - the so called Cognitive Risks for Cybersecurity associated with the Cognitive Hack and Data Poisoning attacks. The cyber battleground has shifted recently from an attack on hard assets to a much softer target: the human mind as human behavior is the new and last “weakest link” in the cyber security armor. The Bruce Schneier’s popular quota: “Only amateurs attack machines; professionals target people” [B. Schneier (2000). Semantic Attacks: The Third Wave of Network Attacks], is now becoming an emerging reality. The cognitive hack takes place when a users’ behavior is influenced by misinformation. In his new book [J. Bone (2017). Cognitive Hack: The New Battleground in Cybersecurity ... the Human Mind, CRC Press], James Bone admits that “the human-machine interaction is the greatest threat in cyber space yet very few, if any, security professionals are well versed in strategies to close this gap”. On the other hand, Cognitive Computing and Collective Intelligence enhance creation of “artificial” decision-makers, agents, cognitive robots, etc. These entities are becoming users of various information systems and sources and automatically learn based on this usage experiences. Therefore, they are also becoming a potential target for Cognitive Hacking. Very recent articles related to security aspects of Deep Learning noticed numerous potential risks of influencing outcomes of Machine Learning (e.g., decision models) by a variety of (training) data poisoning techniques. Therefore one interesting topic for the self-study would be on how to handle such risks (based on system's self-awareness and self-protection) for both human minds and artificial minds (i.e., risks of Cognitive Hacking of the Collective Intelligence) to make future smart systems secure.

 

We will combine overview lectures, self-study, group-work, theoretical and practical assignments and exercises trying to find an optimal approach to everyone.

Recommended reading: Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning, MIT Press, 787 pp.  (http://www.deeplearningbook.org)

 

 

Recommended reading: Michael Nielsen (2017). Neural Networks and Deep Learning. (http://neuralnetworksanddeeplearning.com/)

Deep Learning Resources (http://deeplearning.net/)

 

COMPLETE PACKAGE OF THE SLIDES FOR THE WHOLE COURSE IS HERE

 

Lectures for the course TIES4910-Deep Learning for Cognitive Computing, Theory:

 

·       All the lectures and self-study material in one PowerPoint. (Lectures 1-10):

o   See the lecture slides (download before viewing, enable external content and switch on speakers).

o   Contents:

-         Deep Learning & Cognitive Computing Introduction;

-         Introduction to Neural Networks (for beginners);

-         Neural Networks and Fuzzy Logic;

-         Strengths and Weaknesses of Deep Learning;

-         Recurrent Neural Networks;

-         Deep Learning beyond Deep Neural Networks: What else Can be “Deep”?;

-         Explainable AI and Causality;

-         Deep Learning and Topology;

-         Training Neural Networks (Backpropagation, Overfitting, Regularization, etc.);

-         Competition on the AI/ML-as-a-Service Market (perspectives, fears, and concerns);

-         Online Cognitive Services (they can do almost everything for you);

-         Convolutional Neural Networks;

-         Deconvolution and Style Transfer;

-         Semantic Segmentation;

-         Generative Adversarial Networks and Generative AI;

-         Introduction to Transfer of Learning and Federated Learning;

-         Brief Introduction to Self-Supervised Learning;

-         Attention and Transformers in Deep Learning;

-         Physics-Informed Neural Networks;

-         Short Introduction to Reinforcement Learning;

-         Future of Deep Learning (robots, education, medicine, etc.);

-         Deep Learning and Security: Adversarial Machine Learning;

-         Few old slides as a summary about IBM Watson;

-         Become Master in AI and collaborate with us.

 

·       Additional old slides.

o   Topic: Introduction to Neural Networks and Deep Learning

o   See some old additional lecture slides (download before viewing).

 

·       Additional old slides:

o   Topic: Convolutional Neural Networks for Image Processing

o   See some old additional lecture slides (download before viewing).

 

·       Additional old slides:

o   Topic: Neural Networks with Memory: Recurrent Neural Networks and LSTM Networks

o   See some old additional lecture slides (download before viewing).

 

Recorded Video-Lectures:

The video records for all the lectures can be downloaded and viewed from Moodle

NOTE:  if you have possibility to come to the in-class lectures, choose this option, because lecture content is constantly updated, and the recorded lectures from previous years could be slightly outdated …

 

 

COMPLETION MODE: Your grade will be based on the course Assignment (task for the Assignment is here; deadline: 15 November).

 

How to continue? We recommend also taking the course: TIES4911 Deep Learning for Cognitive Computing for Developers”.

 

 

Collaborate with our research group on developing stronger AI!