Skip to end of metadata
Go to start of metadata

Schedule, agenda, and minutes summary/details for meetings of the Acumos TSC Community Committee.

Weekly Meeting Logistics

Bridge: 

Zoom 2: https://zoom.us/j/9266141474

#acumos-meeting

Tuesdays 9 AM ET

Next Meeting

  will be led by Reuben Klein


Agenda:

  • Confirm Schedule Change for July
  • Update Plan for Documentation
  • Discuss agenda of Release Planning call on July 13 
  • Talk about recording and prioritization of features
  • Review Demeter backlog for Point Release 1 requirements


Meetings

DateMeeting Summary

 

Today's discussion points:

  • Product Committee : ( Community, Architecture & Security) 
  • Components:
    - Model Management: Common Services, On-boarding & Model Deployment
  • - Portal
    - Licensing
    - OA & M
    - Design Studio/MLWB
    - Training
    - Testing
    - Doc - Responsible for all PTLs

 

  • We discussed the planning meeting for July 13 9-11 AM ET
    • Murali Vuppari will send out an invitation outlining the purpose of the meeting and asking for some pre-work
    • We need to discuss updating the Documentation design and how it can be revised to clarify what information applies to the release you are looking at
    • We need to build a significant backlog of features which will be candidates for both Point Releases and Elpis
    • We want to ask PTLs to take a specific role in suggesting important changes and prioritizing suggestions from team members
  • Our next Community Committee meeting will be on July 14 at 9 AM ET
    • Future meetings will be once a month on the second Tuesday of each month.

 

  • We are currently in the election process for a Release Manager - We need to have a manager to lead the planning call
    • Nat is new TSC Chair. He can appoint a candidate for release manager
    • This will have to wait one week.
  • Can we schedule a planning meeting. (Assuming that a release manager is named)
    • Monday 9 AM (Second week of July) may be a good time for that planning meeting.
    • Proposal 9-11 AM (ET) on Monday July 13
    • Complication is that Release 1 planning is already supposed to be done by now (see schedule)
    • Can we quickly bring this project in line with our schedule and then identify features needed in Release 2?
  • Key features we know about which should be in the planning work:
    • Automated Model deployment/launching
    • Improvements to portal UI
    • Security verification for models
    • Licensing functionality must still be completed
    • Revise Documentation to better reflect Demeter-specific design
  • We have set up a feature planning page - people can add their suggestions to that page
    • Discussion of the result on July 13 meeting
    • Release 1 is already in flight and focused on backlog
  • Farheen Cefalu will lead the team in building a description of the O-RAN xApp use case for our next meeting
    • Ken Kristiansen has some content that can be provided
    • Philippe Dooze will assist in the planning of the scenario
    • Guy Jacobson developed the Acumos xApp adapter and will provide additional guidance
    • This could be an end-to-end example (even including a recorded demo) of use case documentation

 

  • Scan this code if you want to join the LFAI Outreach WeChat Group;

  • There are 45 members on this group from China
  • Note that AT&T does not allow employees to use WeChat at this time but others may want to join
  • Christina Harter is looking for 3 more volunteers to help with the June29-July 1 virtual Open Source Summit meeting. Contact info:

Christina Harter, CMP

LF Events & Program Management

The Linux Foundation

T: +1 (817) 308-5838 (PST zone)

E: charter@linuxfoundation.org

Free tickets are available if interested (worth 50 USD)

Virtual booths are available:

https://www.linuxfoundation.org/blog/2020/04/virtual-event-suggestions-for-open-source-communities/

  • How should JIRA items be handled now that we have 3 releases on the schedule?
    • User Stories should be discussed in the Architecture Committee
    • Critical errors should be targeted for .1
    • Low impact errors & simple enhancements can be targeted for .2
    • New features in Elpis
    • Once planned by the Arch committee we can discuss priority in Community and PTL meetings
    • Any new features in Elpis should be captured for the eventual public announcement. This can be reviewed with the TSC.
  • We are still waiting for a review of the backlog items. This hasn't been a priority until now.
    • There are 6-7 items that can be reviewed on the PTL meeting and closed
    • 7 items are planned for implementation as part of the upcoming build.
    • TechM will be committing 1-2 people to Portal Work (onboarding and microservices)
    • Natarajan Subramanian explained that there will be no TechM involvement in ML Workbench.
    • These people will also be responsible for any Licensing work
      • contact Rajesh Pendke for assignment
      • This work has been done by Neeraj Sonar up to now.
    • We need to get a list of contributors from AT&T, Huawei, and Orange for upcoming releases to make feature assignments
      • Ericsson resources are also available to do work, esp on O-RAN integration
      • Nokia should be contacted (probably through Michael Rooke)to identify additional contributors
      • AI4EU is working on some items. Main contact is Martin Welss who works closely with onboarding and microservices team on C/C++ onboarding.
        • We don't know if they want to focus only on that area and we should reach out to them to discuss.
        • There is a possibiltiy of recruiting their students going forward.
        • We need a breakdown of User Stories that need help and find out if there is interest.
        • Philippe Dooze and Guy Jacobson could reach out to them.
  • How do we proceed to get new feature commitments to support our use cases?
    • The O-RAN work should have a lot of interest from all of the current players
    • The key challenge is designing a mechanism to take Acumos models and run them in a 5G RAN, CU or DU.
    • 5G xApp for traffic steering, 5G Spectrum assignment to optimize capacity and RIC Integration are the key use cases.
    • We also need to define how Training would be done for these components.
    • We need to write-up a description of a discussion session dedicated to these topics and circulate them to Ericsson, Nokia, Huawei, Orange and AT&T in order to get focus on these applications
    • There are multiple approaches for incorporation of models into various 5G scenarios.
    • The goal of this meeting would be to define the "What" and the "How" of applying Acumos models to 5G use cases.

 

  • Proposal on scheduling:  Alternate between Community, PTL and Architecture meeting on Tuesday mornings (3 times a month).
  • Question: Is AT&T going to nominate a new LFAI governing board member? Mazin is no longer listed as a member
  • Could SoaJS present to the LFAI TAC for inclusion as a new incubation project?
  • They wanted to understand what is the path to monetization?
  • Better visibility as an LFAI project
  • SoaJS is not limited to the AI/ML space - what is the impact of this to the LFAI?
  • Update on Demeter progress. There is still a UI issue which is holding up the release
    • Hope to have it resolved soon
    • Will be discussed later on the PTL call
  • Need more support for incorporation of LUM APIs in allied projects (ONAP, O-RAN, etc)
    • Have not seen much support from those communities
    • License Management was designed to support many projects, not just Acumos
    • Short term goal is to get support from ONAP via AT&T and Orange as prime movers in this community.
    • Planning to build out LUM as needed to support Virtual Network Functions in a more general way with support from Orange.
  • How do we get people to complete the use cases?
    • Need for succinct documentation on licensing, install, configuration, federation, etc.
    • Today you need 10 people just to figure out how to do something.

 

  • Waiting for release document to be completed.
    • Release quotes has been received from ZTE, Orange, TechM & Nokia
    • Waiting for AT&T, Ericsson, & Huawei quote
    • Waiting for sanity test to be completed.
    • Image will be moved to release repo and a Demeter branch will be created.
  • Originally planned for last Friday, new deadline will be discussed on PTL call today
  • Suggestion to have the Community, PTL & Architecture meeting every alternate week. 

 

  • TSC is looking for a 2 line statement on the Demeter Release from this team: Voting indicated that people are most interested in highlighting Cloud Enablement and the Bidirectional Federation feature for improved Training.
    • We will send a 2 line statement to the LFAI marketing team for inclusion in the announcement.
    • Reuben Klein will send in a suggested statement.
  • Schedule - Are we comfortable with the current schedule?
    • Seems to work well.
    • We will have an extended Community meeting on 6/2 (merged with PTL meeting) where we can discuss priorities
  • Murali Vuppari will set up a notes page for submission of target user stories.
  • Because this is a point release, we will focus on completion of work already in progress and bug fixes.

 

  • Due to a conflict with the LFAI Governing Board meeting schedule, this team will skip the first Tuesday of every month
  • No one has self-nominated for the TSC position - confirm with Anwar that this position will be left open temporarily
    • We should encourage team members to join. Perhaps weekly TSC meetings are too often
    • Going forward the TSC should not be a weekly scrum of scrums call.  It should be much more high level dealing with Major Issues, Dependencies, etc.
    • The TSC could focus on assigning work to other committees (Arch, Community, PTLs, etc.)
    • If the TSC is going to get down in the weeds then we should not have all these other meetings.
    • If the TSC is going to function as an executive committee then the other chairs need to take responsibility to present status and issues to the TSC for a vote.
  • Question: Can we move away from a structured development process to more of a self-driven agile style of project
    • We would need to be defined in a more modular way
    • This would allow groups to work independently in parallel
  • Redefining the modules is something that could be done
    • Jessica Kim pointed out that we have not grown as a community over the last two years. Do we want more participation? How do we do that.
    • Reuben Klein brought up that perhaps the role of the sponsor companies is too controlling
    • Natarajan Subramanian said we are not getting fresh ideas
    • Parallel branch development requires some restructuring
    • We would need to spin up dev and test environments for each active branch - which could be done, but requires support from the TSC
    • But it would be rather easy to begin more of the "Design Thinking" stage of the major release while dot releases are being developed.

 

  • Jessica Kimspoke with Tencent senior leadership: Will Yang - GM of AI Platform, Technology Eng'g Group (TEG) and Haobo Fu LFAI Delegate
  • Sponsorship of Acumos public cloud
  • Sponsorship of LFAI Technology Showcase Day for APAC (in August)
  • April 23, 2020 TAC Open Data Source Initiative for Covid-19
  • Haobo Fu was in attendance at today's meeting
  • Open Source summit (scheduled for Austin) will now be a virtual event
  • Discussion of features and Demeter release
    • Scheduled for June 29
    • As this will no longer be a live event, we don't know what will be done instead of exhibition booths
    • LFAI was scheduled to host a booth this year.
  • Demeter is scheduled to be released the week of May 22
    • We expect announcement on May 25
  • We need a schedule for the discussion of Elpis planning cycle
    • We still don't have a release manager for any of the upcoming releases (D.1, D.2, E.0)
    • Role: working with the community on the feature commitments and coordinating with the PTL team
    • Report progress to the TSC on a weekly basis
    • This is a feature tracking role - not really project management
    • Opportunity for leadership in the planning process
    • Coordinate between the various teams
    • Anyone willing to take on this responsibility should bring themselves to the attention of the TSC
  • Review of Demeter Release messaging
    • I will post a link to Demeter page containing the release deck
    • We will take the backlog list of 470 JIRA items and classify it according to the component impacted
    • These will be handed out to the respective PTLs for recommendation
    • PTLs should eliminate all the no longer relevant items/duplicates (clean up)
    • Submit a prioritized list for consideration in Elpis planning
    • The community committee will then review these recommendations.
    • This should be completed during the next month to coincide with the timing of the Demeter release.

 

Proposed to move this meeting to 9:00 AM ET in order to get more folks from China and/or Europe

100% of participants voted yes 

Reuben Klein will update the calendar in Lists.io - Jacqueline Serafin is a good contact to ensure we are doing it right.

Release schedule proposal from Natarajan Subramanian

Sprint 4 will end in May

The project will Issue 2 add'l maintenance releases on Demeter

Propose that we plan to release Elpis around June of 2021

Trying to do one major release each year

Trying to avoid a tight schedule during the low productivity months (Aug and Dec).

Plan for maintenance releases to take 90 days

for Demeter.1 and Demeter.2

Is this in conformance with LFAI policy? (2 rels per year is the norm, this would be 3 releases 1 major and 2 dot releases)

Reuben Klein can raise this with LFAI TAC

We should influence the TAC to limit project acceptance to those projects which are willing to build interoperability with other projects in LFAI

Reuben Klein is the new AT&T TAC member

Demeter ETE1/2/3 ends this Friday  

Demeter ETE 4 will be a two week sprint

Starts next Monday  

Focus on critical defects. No new features.

Complete by  

We will begin working on branching on  

Demeter announcement should be by  

Jessica Kim raised the issue that no nominations have been submitted for TSC Chair by the deadline. What should we do?

Anwar suggested "taking down" the nomination. This must mean postponing the election for a time.

Normally the vote is extended for one week in this event.

TSC chair must be the first election of the year, so this is a critical milestone.

In addition to current backlog, Farheen Cefalu would like to submit a series of UI user stories that should make a big diffference.

Murali Vuppari suggested opening all necessary Epics in JIRA but do not assign them.

This will put them into the backlog for the next release

The license management PoC revealed a number of features that are needed for the License Management plan to work

Michelle Martens will document these. They should be prioritized for the next release.

 

Should we change the schedule of the Elpis release?

There are dead periods (summer months and December-January) that make it difficult to get anything done.

Perhaps we look at 9 months as a schedule goal (March instead of November after a 6 week break at end of year?)

Suggestion:

add 2 months to Demeter maintenance release (Demeter.1 defects primarily) and then

begin Elpis and have it run the next 6 months.

Patch releases (Demeter.2 for high visibility defects/no new features) could be on an independent schedule running in parallel to the next major release

Elpis could focus on resolving the backlog

Murali Vuppari can review some of the backlog and identify which JIRA items are relevant and which are not.

Then we can prioritize the important stories

This must be completed in May

Another important area is to complete the migration to cloud service

Try to get feedback from the community on what to emphasize

Identify opportunities to team up with other LFAI projects

Integration of ONNX requires more Python resources

Import ONNX objects

Enhanced model runner for ONNX

Introspection to create protobuf

Create ONNX microservice

We should pick this up as an opportunity to increase the capabilties

How does ONNX runtime fit in? Is it part of ONNX O/S or a separate project?

Common development/test environment for the community

Currently only TechM and AT&T can work in a common environment

Our Azure development environment will go away at the end of 2020

We need to find somehting more universal and available to the entire community

perhaps Msft would be willing to donate an environment

They are a supporter of the ONNX community

We would need to make sure that Acumos does not become an Azure specific tool

We are still in need of a Release Manager to take over for Natarajan Subramanian

Looking for volunteers

This is an appointed role

Can Huawei, Ericsson, Nokia, Orange step up? There are other companies in LFAI, but we don't see any interest from them.

  

 The name Elpis has been chosen for the "E" Release: Release Page

We will list requirements on this page - there is a large backlog built up in the last 3 releases. This backlog should be scanned for Elpis Epics

Demeter release is scheduled for May - Elpis work will begin around Memorial Day (end of May)

Nat is not able to do the release management so we will need a volunteer to lead this effort

This is not an elected position - it must be someone with the experience and time to do this job

We should draw from other LFAI efforts and teams to enhance Acumos features

MLFlow - Model target management

SoaJS - CI/CD (is it easier to improve our Jenkins scripts than to incorporate SoaJS?)

We need a way to organize a discussion of alternatives and strategy. Many members of the community have experience with a range of tools

This discussion can then be used to set up Elpis requirements

The Elpis release should not be too ambitious from a feature perspective

Less development/more integration

We need to organize leadership on this to provide a direction

We need experience in potential tools which are integration candidates - this should not be a wide net. We want consensus around one-two candidates

Less interested in the perfect component than one that covers 60-70% of the functionality. but cuts our development time significantly

Michael Rooke suggests that we concentrate on Marketplace aspects of Acumos

Reuben Kleinthinks deployment to multiple environments with CI/CD is important

Demeter Sprint 4 is scheduled to complete this week

Workbench team is somewhat delayed (2 weeks?)

License management is somewhat behind as well

SoaJS is also an issue at this time

We will not have to move the release target date, but the testing schedule will be impacted.

If too many items come in at the last minute it may create an unsustainable timeline

We may have to compromise on features if we cannot maintain our schedules

Schedule time at next meeting to assess where Demeter stands after completion of Sprint 4

Use Case Leads need to get a description and scope written up

Please go to the Use Cases page and provide a good additional details

Follow the Link to your use case and add a clear description of a process that we want to demonstrate

This should describe an end-to-end scenario which captures a complete function of Acumos (not just a feature)

The Scope should lay out the steps of the use case and what they accomplish

The Timeline should lay out objectives culminating in a demo to the Community Committee at the end of May (when Demeter is released)

The demo will be recorded and put onto the Wiki as a training item.

The Dependencies section should identify the other use cases, JIRA items and resources that you are depending on to do the demo

You can also show which other use cases depend upon this one.

  

Why are we building these use cases?

The idea here was to create an end-to-end scenario which demonstrates a meaningful application of Acumos.

This should demonstrate a real world value and not just an abstract feature.

The process identifies all of the features needed to accomplish a real-world task.

It demonstrates that each dependent feature cited in the use case works correctly

It demonstrates that the individual features that have been built all work together

This reestablishes a practice that was implemented pre-Athena in which end-to-end scenarios were created describing not just features but end-to-end flows

These use cases can be use as demos of the new capabilities introduced with each release

Ideally there should be use cases that incorporate all the priority features in the release.

If not, then why were those features considered high priority? They ought to have a real-world value that can be demonstrated.

The list of approved use cases have been compiled on a single wiki page. each with a link to an individual Use Case Project Page

We reviewed the Use Case project pages and identified the sections that need to be completed

It is the responsibility of the Team Lead to assemble a team and list them on the Project Page

It is the responsibility of the team to identify the scope of the use case and set a schedule for completion of milestones

We will send out email to the teams that have been identified asking them to fill in the pages as they see fit

They should set up team meetings and teams should report progress from time to time to the Community Committee

This can include demonstrations of the use case as far as it has been completed

It is the responsibility of the team to identify any missing dependencies and set up JIRA items accordingly

Each project should be reviewed with the Community Committee at least at inception and at completion

We will review with the TSC to find out whether linkage of release dates and use case completion should be made.

 

Michelle Martens spoke about RTU editor and applying it to Acumos - can portal launch the RTU editor?

Should we have a License Admin role?

The Lic. Admin would be responsible for setting the RTU information in the license agreement as well as other terms and conditions

We need a way to restrict access to license administration screens

Michelle Martens will bring this up in the Architecture Committee meeting

Farheen Cefalu pointed out that Acumos allows role definition and assignment now, so this does not require a new feature

We need a user story which tests/exercises this role

Work with the portal team to get this feature incorporated

As today is a holiday overseas, attendance was limited today.

We will postpone additional discussion until the next meeting

 

Planning for the Use Cases, above, should move forward to include complete descriptions, team definition, milestones, schedule and status

We will set up a template for a Use Case wiki page for each one.

Reuben Klein and Michelle Martens will work together to define this template using Use Case 4 as an example

We will ask the Use case leads to fill these pages out and to walk the Community Committee through their plans at upcoming meetings

Reuben Klein will send out an email to all the leads to explain what is needed.

 

We made a recording of this meeting and have attached it here: Community Meeting Recording 1-28-20

We reviewed the general categories of use cases and why we are proposing them

We wish to establish the value of machine learning in contrast to predictive analytics

Exploit the generality of ML when applied to different datasets

Create a library of good models for adopters to make use of as a jump start

Identify product gaps that result from the difficulty of (if any) of tying together multiple predictors

Identify product gaps in the areas of training and lifecycle.

We asked anyone who would like to suggest a use case or who would like to build on of the defined use cases to contact the moderators (Natarajan Subramanian or Reuben Klein )

We discussed MLFlow

The advantages of using existing opensource projects to accelerate Acumos development with fewer resources

The benefit for the LFAI community which is derived from closer interoperability between Projects.

Allow users to adopt a full set of AI components which are known to interoperate.

We watched a video demonstration from the MLFlow team

Reuben Klein showed slides from an MLFlow presentaion attached here: MLFlow.pdf

 

The TSC has sent out a request for nominations for a Chair for the Community Committee.

Natarajan Subramanian has sent out an email to the distribution list as well.

Looking for a response by the end of this week.

Nat will send out a reminder if we have no nominees before the next TSC meeting.

Syed Anwar Aftab suggested we contact other people directly to ask them if they are interested in taking this role

How do we get better focus and more interest in the Community Committee?

We should continue to discuss new desirable features and designs

The focus ofo this meeting should be on use cases and user experience

List areas where we are discussing use cases from the Demeter meeting

Use case for Bidrectional communications: An exception occurs on a model running in a partner site - exception data transmitted to publisher. (review with Wenting Sun )

Monetization of model development: Build models, advertise them and sell/support licenses

Michelle Martens brought up participation on sub-commitees and commtment of resources from the Community

What can be done to encourage additional participation?

We need to have incentive for the community to contribute and commit to Acumos

We could ask the LFAI community which is doing successful outreach to make interoperability with existing projects (such as Acumos) a requirement for graduation of any LFAI incubation project (e.g. ONNX, Horovos, Milvus, etc.)

 

Discussed several use cases:

Network Disruption Predictors:

Given a fault condition, predict severity (i.e. services impacted, scope and timing of any service disruption) at a particular location based on available logs.

VNF Failure Predictor: Automatically analyze and detect early signs of anomalies of VM behavior in the network. Proactively predict potential faults or failure of VMs in the network. (Could take ML models already developed and apply to ONAP)

Infrastructure Predictor: Predict resource utilization at a given future point in time based on current performance trends using a historical context (May extrapolate node occupancy or load from current trends. Could factor in busy periods and cyclic volatility)

Automated planning and purchasing; Learn when to repair or order replacement equipment based upon an increased likelihood of service impacting failures in the network.

Network Management:

Learn to identify unusual traffic patterns and compensate (adapt flows around failures/overloads etc.)

ML Enabled Service Assurance (Tingting’s project)

Self-healing of network failures

Report or improve performance of a network resident application by adjusting network parameters.

Customer Care:

Improve precision of technician dispatch

Improve proactive care – generate contact information for impacted customers using subscription data

Third Party Application Support:

Enterprise Machine-to-machine applications

Edge Router monitoring for traffic anomalies that may indicate intrusion

Use Cases that can justify Acumos work and specific features

IoT device applications - e.g. webcam monitoring of network equipment, home security,

factory/assembly line monitoring  - predict failures. schedule maintenance, profit/loss calculation. scheduling, capacity management

Boreas/Clio Migration - People need to update licenses (republish models) when Boreas models are run in Clio

AT&T is supporting a PoC on Clio to demonstrate that license management actually works in production

Working with Ericsson licensed models and TechM licensed models

 

We lacked a quorum to get a good concensus on how we want to work with TM Forum

Several use cases were discussed and new ones suggested

Reuben Klein will provide feedback to TM Forum on a preliminary list of use cases and objectives

We will wait for a larger audience to finalize this work

As the next 2 weeks are holiday weeks, our next meeting will be on Jan 7

 

Continued discussion setting up the Demeter Release Project Status JIRA page that was begun in the PTL meeting

Murali managed the discussion and updated the page cited above

Bryan discussed OA&M EPICs

Sayee discussed MLWB EPICs

 

Discussed value statement - Business value of Demeter priority features

PTLs to complete these statements

Discussion at next week's meeting on each

Seeking Community agreement on value of each priority item

We need review of ACUMOS-3773

Should be ready by this Friday and could be closed

Check at next call if we are done with this, or discuss further as necessary

LUM data accuracy validation

RTU data accuracy  - ACUMOS-3639

 

Demeter epics are due Nov 22

User stories done by Nov 29

We did not have a quorum to discuss ACUMOS-1040. We will need a meeting dedicated to discuss this and understand what is required.

A design is required for this feature. Propose to create a design document and vote on that.

Reviewed LUM demo - but only went through about 50% because of time and questions

Acumos LM Clio - Demo

We need to have a review of the detail with security SMEs

If we were to have an automated way of creating trust in any process, what would we use?

LUM is basing artifacts on Acumos identifiers. Is there a validation process that can confirm these identifiers in the LUM?







 mm-dd-yyyy 

No meeting due to Lockup Demeter Planning meetings Wednesday-Friday.

See lockup details at this link.


next release and the requirements has to be aligned to the success criteria.


Manoop:/Farheen : Community committee gather the requirements and discuss it with Architecture committee to finalize it.

Karthic and Sachin presented the AutoML concept. Karthic shared the CLMP work in progress AutoML solution. We need to further discuss in which project AutoML should be developed. Reuben suggested on of the following: model onboarding, model design or training. We agreed that AutoML is an important candidate for Boreas release.


Bryan presented the challenges we have as a community – what are the expectations we have about the trust we put on the models.






  • No labels