Service Computing Benchmark Hub
HSC-Bench: A Unified Benchmark for Service Recommendation and Service Composition
HSC-Bench is an open-source benchmark for service computing. It provides standardized datasets, evaluation metrics, baseline implementations, leaderboards, and HSC+, a large-scale AI model service dataset constructed from Hugging Face services.
Motivation
Why HSC-Bench?
HSC-Bench addresses fragmented datasets, inconsistent protocols, separated task evaluation, and weak service reproducibility in service computing research.
Fragmented Data
ProgrammableWeb, QWS, WS-Dream, HSC, MovieLens, and Amazon differ in fields, preprocessing, and task assumptions.
Inconsistent Evaluation
Different metrics, candidate sets, and splits make fair comparison difficult across published models.
End-to-end Gap
Recommendation and composition are often evaluated separately although candidate quality affects workflow quality.
Reproducibility Issues
Expired links, inactive services, and non-repeatable QoS measurements reduce long-term benchmark value.
Supported Tasks
Recommendation and composition under one protocol
Dataset Overview
Dataset families covered by the hub
| Dataset | Tasks | Domain | Main Fields | Notes |
|---|---|---|---|---|
| ProgrammableWeb | Service Recommendation / Service Composition | Web API | Mashup, API, invocation relations, tags, descriptions | Classic Web API recommendation and composition dataset; platform availability may limit reproducibility. |
| QWS | Service Composition / QoS Optimization | Web Service | Response time, throughput, availability, reliability, cost-like QoS attributes | Classic QoS benchmark for service composition and multi-objective optimization. |
| WS-Dream | QoS Prediction / Service Recommendation / Composition | Web Service | Response time, throughput, user-service invocation matrix | Useful for QoS prediction and recommendation; functional semantics are relatively weak. |
| HSC | Service Recommendation / Service Composition | AI Model Service | AI model services, service workflows, QoS, function tags | Hugging Face based AI service composition dataset. |
| HSC+ | Service Recommendation / Service Composition / QoS | AI Model Service | Function tags, input/output parameters, QoS, requirements, workflows | Core dataset of HSC-Bench for unified service computing evaluation. |
| MovieLens | General Recommendation Baseline | Recommender System | Users, items, ratings | Used to validate generalization of recommendation baselines. |
| Amazon | Cross-domain Recommendation Baseline | E-commerce | Users, products, reviews, interactions | Used for cross-domain recommendation baseline comparison. |
Leaderboard Preview
Static first, protocol-driven from day one
The first version uses YAML/CSV-backed tables and manual pull requests. It can later evolve into automatic online evaluation.
Recommendation Top Models
- SRLCFHSC+ · Benchmark Model
- MTFMHSC+ · Neural
- GSATHSC+ · Graph-based
Composition Top Models
- GNNPN-SCHSC+ · Learning-based
- SDFGAHSC+ · Optimization-based
- DAAGAHSC+ · Optimization-based
HSC+ Highlight
One dataset for recommendation, composition, and QoS analysis.
HSC+ is built from Hugging Face model services and enriched with functional annotations, QoS measurements, realistic user requirements, and executable workflows.
Explore HSC+Team & Publications
Maintained as a long-term research asset
Find maintainers, supervisors, group information, representative papers, BibTeX entries, and contact channels.
View Team & Publications
