TMK Modeling Progress

At this point, I’ve already submitted my TMK json files as a pull request (PR) after being peer reviewed by two developers (Erin and Shashank) and it’s now time to test it. Github commit: The following is in my task ticket, I am currently blocked in testing my json files since the server is currently migrating from the MCM architect’s own AWS account to the ALOE DILAB IVY’s AWS account.

• Task 1 [Completed] Create TMK model for Means-end analysis skill – Available in DILab IVY dropbox
• Task 2 [Completed] Peer review TMK model – Peer reviewed with Erin and Shashank, created branch from ivy-main #21
• Task 3 [Ongoing] Sanity test TMK json files with MCM – Issues in running IVY MCM server

MCM Backend Tasks

Since being assigned to this new role in the lab, I have been looking for ways to improve the current architecture. The following is my proposed architecture for the cloud infrastructure:

Customer interfaces with Amazon Route 53 where health checks can be established while the EC2 instance (private subnet) and database (data subnet) which is enveloped by a private VPC. Only Route 53 and the EC2 instance is interacting so the private IP for the EC2 instance is not exposed to the public.

In the next sprint, I proposed the following features:

• Automated Github Actions Integration for CI/CD pipeline – Push-based with Github actions runner (https://austingil.com/automatically-deploy-from-git/)
• System Security: Implement Health Checks (CloudWatch for AWS)
• System Security: Security Groups and Firewall Configuration

The following document is my proposed CI/CD Methodology:

• Proposed CI/CD Methodology

TMK Evaluation Metric

When evaluating the explanations generated by the IVY MCM infrastructure using TMK files, I believe that the metrics of completeness and comprehensibility should be given the highest priority. Completeness is crucial because incomplete responses can result in significant gaps in knowledge, misconceptions, or confusion. An educational tool loses its value if it fails to provide thorough and accurate information, as this can lead to further misunderstanding rather than enlightenment. Ensuring that responses are complete means that learners receive all necessary context and details, which is vital for fostering a deep and accurate understanding of the subject matter.

Comprehensibility is equally important because the educational value of the material is significantly diminished if it is difficult to understand. Clear and easily understandable explanations make the content accessible to a broader range of learners, including those for whom English is a second language. By prioritizing comprehensibility, we ensure that our educational tool can effectively support diverse learners, enhancing their engagement and retention of the material. Therefore, in the context of the IVY MCM infrastructure, completeness and comprehensibility must be high-priority metrics to maximize the educational impact and utility of the generated explanations.

My rankings:

• High: Soundness, Completeness, Coherence, Comprehensibility
• Medium: Confidence, Compactness, Contrastivity
• Low: Translucence, Anthropomorphism

DILab Meeting

During the meeting, Rahul Dass presented on the development of our minimum viable product (MVP) for the machine learning-based question-answering system, focusing on skill classification and roles and responsibilities. He outlined the plans for the MCM, including the alpha release featuring TMK models scheduled for October 7, which will demonstrate these models with specific user scenarios. Rahul proposed alternating 2-week sprints dedicated to demos and MVPs, seeking input on roles and evaluations.

My task centered on testing my TMK files, working on the MCM Database backend and the upcoming sprint. I expressed confidence in meeting the next week’s deadline during a discussion on project feasibility with Rahul and Shashank. Spencer R. suggested organizing sprints around user-visible features supported by infrastructure requirements. In this context, Rahul asked the team to rank metrics for use cases, leading to a debate on using numeric values versus a Likert scale. Erin proposed a 1-4 scale for metrics, which spurred a group discussion on the best evaluation methods.

The summer reading group has been cancelled due to conflicting schedules of the presenters.