TMK Modeling Progress

As advised, I tried decomposing the method model even further. The method model was created to define the specific operations and transitions involved in comparing a given state to a goal state, detailing the steps for breaking down block positions, extracting individual positions, comparing these positions, and counting differences. The task model refers to this method model by specifying the conditions, inputs, outputs, and the corresponding methods to be used for each task, ensuring that the transitions in the method model align with the task names. On the other hand, the method model was created to outline sequential operations based on tasks, forming a simple one-directional state machine.Finally, the knowledge model was constructed by first identifying the individual instances, representing each block in both the goal and given sets. Triples were then created to define the possible positions of blocks in relation to each other and to surfaces. Finally, concepts to encapsulate the overarching categories of blocks, states, and surfaces, while relations were to interconnect these concepts together.

• Task Model v1 json file
• Method Model v1 json file
• Knowledge Model v1 json file

After my sync with Erin, I’ve realized I needed to tie the TMK model back to the skill that the learner is needing to acquire. This means that the knowledge model will have the concepts for means end analysis and how it is achieved through the block world problem. I also needed to edit the knowledge model so that it will have the specific wordings that the problem is using. This was emphasized last team meeting where standardization is one of the key goals this semester. Similarly, I’ve written 1:1 mapping between tasks and methods, Erin has mentioned that a task can have multiple methods and suddenly I start thinking about this problem in a different manner. I’ve mentioned that I will approach this with two tasks: (1) Evaluating the means end difference between given and goal state and then (2) Choosing the best block arrangement that will satisfy the means end analysis. From there, I can create methods that will achieve this task. The method model seems to be the most complicated one to create since I have to think about the state transitions between them. She also recommended looking at the current Block Problem presentation and how state transitions were expressed and go from there. Her feedback gave me multiple things to work on.

DILab Meeting and Knowledge Share

During the meeting, Rochan proposed rewriting the Task-Method-Knowledge (TMK) representation to improve its hierarchy and structure, emphasizing the need to link task and knowledge components hierarchically and to structure keywords comprehensively within the TMK system. Rahul discussed the potential of multimodal models in natural language processing to enhance the hierarchical representation of task-oriented knowledge, noting that the IVY framework currently limits knowledge questions to the baseline models while using different models for method and task questions. The discussion also covered challenges in mapping tasks to methods, with Rahul highlighting difficulties in one-to-one mappings and the need for ranking methods. With this in mind, Erin and I have decided to keep the Task and Method 1:1 mapping the same. I will still work on making sure that the high level skill is reflected on the TMK model next.

Sashank presented on Iterative Task Learning (ITL), explaining its goal of enabling machines to learn general tasks through natural language interactions. He described a hierarchical task network (HTN) used by a digital assistant to decompose tasks into smaller subtasks. The HTN is built through interactions, with algorithms integrated into a central interaction manager to perform multi-step reasoning. These algorithms interpret natural language by segmenting sentences into commands, mapping them to actions, and verifying their validity. The process includes identifying arguments using word knowledge, reconstructing sentences for accuracy, and employing error checking and paraphrasing. This has lead me to look for another approach that can also accomplish the same thing, being problem oriented rather than task oriented (see literature review below).

DILab Summer Reading: Transformers

I started to take a stab at Chapter 12: Transformers in “Deep Learning: Foundations and Concepts” by Christopher Bishop. I learned about the concept of attention, transformer operation, self-attention, and soft attention.

Concept of Attention

The concept of attention in neural networks is important, particularly in predicting the next word in a sequence by heavily relying on certain key words. Unlike standard neural networks, where the weights are fixed once the network is trained, attention mechanisms dynamically adjust weighting factors based on the specific input word or data. In natural language processing (NLP), words are mapped to vectors in an embedding space, capturing their semantic meaning.

Basic Transformer Operation

Transformers operate on a data matrix, XX, with dimensions N (tokens) x D (features). This matrix represents a set of input tokens. The fundamental building block of a transformer takes X as input and produces a transformed matrix X~. Transformers consist of multiple layers, each with its own weights and biases, which are learned using gradient descent via a cost function. Attention coefficients play a crucial role in this process. Input tokens x1 ,x2 ,…,xn in the embedding space are transformed into output tokens y1 ,y2 ,…,yn , with yn being a linear combination of input vectors, weighted by attention weights anm . These weights determine the influence of each input token on the output token, with significant tokens receiving higher weights.

Concept of Self Attention and Soft Attention

Self-attention, an important mechanism in transformers, determines these weights by computing a measure of similarity, such as the dot product, between tokens. Soft attention, which uses continuous variables for matching queries and keys, allows for differentiable computations suitable for gradient descent. The softmax function is used to normalize these weights, ensuring they form a valid probability distribution. This mechanism allows the model to effectively capture dependencies and relationships within the input sequence.

DILab Summer Reading Discussion

Steven Bryant discussed the capabilities and limitations of generative AI, particularly transformers. Steven demonstrated how transformers can be used to create music and videos, while Dr. Ashok raised concerns about their consciousness and intelligence. The discussion also delved into the mechanics of transformer models, attention mechanisms, and their impact on sequential data. This raises questions about the trustworthiness of implicit representations and the power of attention in changing word representations and relationships. I was also quite pleased that we talked about the limitations of LLMs. What stood out to me was the “token problem”, so once an LLM generates a token, it cannot go back and fix it if it turns out to be incorrect. Thus, this makes LLMs prone to propagating errors.

Literature Review on LLM-based Cognitive Agents for Problem Specification

In exploring the implementation of a Large Language Model (LLM) enabled task analyst agent, this literature (“Eliciting Problem Specifications via Large Language Models” by Ray et al. ) review delves into how LLMs can be leveraged beyond their traditional role as natural language generators to produce detailed problem specifications. The primary aim is to solve instances of problems within the same problem class and potentially disintermediate the problem formulation process. The agent operates by using prompts designed around problem spaces, extracting task knowledge from the LLM. This approach differs from Interactive Task Learning (ITL), where an LLM agent generates descriptions of operators and their effects, rather than learning through human interactions. The design of the agent follows four high-level directions: defining characteristics of the problem, defining characteristics of problem-solving, formulating the problem space, and assessing the quality of the solution. Key definitions include a problem as a set of initial states, goals, and path constraints, and a problem space as a set of symbolic structures and operators. The review also mentiones concepts such as translating human knowledge into AI system representations, the Agentic LLM approach, and methods like Tree of Thoughts (ToT) which use natural language rather than formal representations.

To me this approach is interesting because of how DILab’s approach for cognitive agent is on the side of ITL implementation. Both share the goal of enhancing Cognitive Agent’s task understanding and execution through natural language interactions. However, they differ in approach and focus. ITL emphasizes constructing hierarchical task networks (HTNs) that decompose tasks into subtasks, using algorithms for multi-step reasoning, error checking, and direct interactions to refine task performance. In contrast, the LLM-enabled task analyst agent focuses on generating detailed problem specifications from structured prompts, leveraging LLMs to extract task knowledge and create comprehensive problem spaces. The LLM-enabled task analyst agent focuses on being a bit more independent from human intervention and interaction. This made me think of exploring the use of problem specification modularization to implement transfer learning, where Agents can use the same problem solving methods for problems in the same classification.

Literature Review on ICAP Theory

The cognitive outcomes from knowledge change processes vary with the level in the ICAP Theory. For passive learning, understanding is minimal, while for active learning, it remains shallow. Constructive learning, however, enables knowledge revision, linking with other schemas, and self-explanation, enhancing the potential for transfer. In the procedural domain, this allows procedures to be transferred and applied in new contexts. Our goal for the cognitive agent IVY is to learn and teach skills and transfer them to problem-solving contexts, such as moving from Guards and Prisoners to solving Raven’s Progressive Matrices. As the authors acknowledge, this is challenging, but it remains our aim. Finally, interactive activities, through dialogue, expand both peers’ knowledge and skills, leading to innovations more powerful than independent learning. This deep understanding is what we aspire to achieve with human-computer interaction, fostering rich and beneficial learning and skill transfer between students and cognitive AI agents.

Other Reading

Since I got to Porto, of course I had to pick up a book at the world famous Livraria Lello, so I started reading Animal Farm by George Orwell. Yes yes, I will need to pick up my other book but I have now been enticed in completing and contemplating this satirical allegorical novel for entertainment. I learned from the How to Read book to classify each book you are about to read. My purpose here is to be entertained and likewise correlate the allegory to a more humanist and sociological point of view. During the train ride, I have read the chapter in How to Read a Book: The Classic Guide to Intelligent Reading about the importance of being a demanding reader and pigeonholing a book. I also tried skimming some bits on the mentioned pages for Plato from the index page, a technique for pre-reading and gaining some structural and conceptual idea of the book mentioned by the very same book.