이 문서에서는 OpenStack Cinder 서비스에서 볼륨을 생성하는 내부 프로세스를 상세히 분석합니다.
Cinder API의 진입점은 cinder/api/v3/router.py 파일에 정의되어 있으며, 볼륨 관련 요청은 cinder/api/v3/volumes.py의 VolumeController로 라우팅됩니다. 볼륨 생성 요청은 다음과 같은 코드로 처리됩니다:
def create(self, req, body):
# ... 파라미터 검증 및 처리 ...
new_volume = self.volume_api.create(context,
size,
volume.get('display_name'),
volume.get('display_description'),
**kwargs)
retval = self._view_builder.detail(req, new_volume)
return retval
여기서 self.volume_api는 cinder.volume.api.API() 인스턴스이며, 실제 생성 로직은 cinder/volume/api.py에 구현되어 있습니다:
def create(self, context, size, name, description, snapshot=None,
image_id=None, volume_type=None, metadata=None,
availability_zone=None, source_volume=None,
scheduler_hints=None, source_replica=None,
consistencygroup=None, cgsnapshot=None, multiattach=False,
source_cg=None, group=None, group_snapshot=None,
source_group=None, backup=None):
create_params = {
'context': context,
'raw_size': size,
'name': name,
'description': description,
'snapshot': snapshot,
'image_id': image_id,
'raw_volume_type': volume_type,
'metadata': metadata or {},
'raw_availability_zone': availability_zone,
'source_volume': source_volume,
'scheduler_hints': scheduler_hints,
'key_manager': self.key_manager,
'optional_args': {'is_quota_committed': False},
'consistencygroup': consistencygroup,
'cgsnapshot': cgsnapshot,
'raw_multiattach': multiattach,
'group': group,
'group_snapshot': group_snapshot,
'source_group': source_group,
'backup': backup,
}
try:
sched_client = (self.scheduler_rpcapi if (
not cgsnapshot and not source_cg and
not group_snapshot and not source_group)
else None)
volume_client = (self.volume_rpcapi if (
not cgsnapshot and not source_cg and
not group_snapshot and not source_group)
else None)
workflow_engine = create_volume.get_flow(self.db,
self.image_service,
availability_zones,
create_params,
sched_client,
volume_client)
except Exception:
msg = _('Failed to create api volume flow.')
LOG.exception(msg)
raise exception.CinderException(msg)
볼륨 생성 프로세스는 TaskFlow를 사용하여 워크플로우를 구성하며, cinder/volume/flows/api/create_volume.py의 get_flow 함수에서 다음 작업들을 순차적으로 실행합니다:
def get_flow(db_interface, image_svc, zones, creation_params,
sched_api=None, volume_api=None):
"""API 엔트리포인트 워크플로우를 구성하고 반환합니다.
이 워크플로우는 다음 작업을 수행합니다:
1. 의존성 있는 키와 값을 주입
2. 입력 파라미터 추출 및 검증
3. 쿼타 예약 (실패 시 롤백)
4. 데이터베이스 항목 생성
5. 쿼타 커밋
6. 스케줄러 또는 볼륨 매니저로 RPC 호출
"""
flow_id = ACTION.replace(":", "_") + "_api"
api_workflow = linear_flow.Flow(flow_id)
api_workflow.add(ExtractVolumeRequestTask(
image_svc,
zones,
rebind={'size': 'raw_size',
'availability_zone': 'raw_availability_zone',
'volume_type': 'raw_volume_type',
'multiattach': 'raw_multiattach'}))
api_workflow.add(QuotaReserveTask(),
EntryCreateTask(),
QuotaCommitTask())
if sched_api and volume_api:
api_workflow.add(VolumeCastTask(sched_api, volume_api, db_interface))
return taskflow.engines.load(api_workflow, store=creation_params)
각 Task의 역할은 다음과 같습니다:
- ExtractVolumeRequestTask: 요청 파라미터 검증 및 처리
- QuotaReserveTask: 쿼타 확인 및 예약
- EntryCreateTask: 데이터베이스에 볼륨 레코드 생성
- QuotaCommitTask: 쿼타 최종 커밋
- VolumeCastTask: 스케줄러로 RPC 전송
스케줄러는 cinder/scheduler/rpcapi.py를 통해 요청을 수신합니다:
def create_volume(self, ctxt, vol_data, snapshot_id=None, image_id=None,
request_spec=None, filter_props=None, backup_id=None):
vol_data.create_worker()
client_ctx = self._get_cctxt()
msg_params = {'snapshot_id': snapshot_id, 'image_id': image_id,
'request_spec': request_spec,
'filter_properties': filter_props,
'volume': vol_data, 'backup_id': backup_id}
if not self.client.can_send_version('3.10'):
msg_params.pop('backup_id')
return client_ctx.cast(ctxt, 'create_volume', **msg_params)
스케줄러 매니저(cinder/scheduler/manager.py)는 또 다른 TaskFlow 워크플로우를 실행합니다:
@objects.Volume.set_workers
@append_operation_type()
def create_volume(self, context, volume, snapshot_id=None, image_id=None,
request_spec=None, filter_props=None, backup_id=None):
self._wait_for_scheduler()
try:
flow_engine = create_volume.get_flow(context,
self.driver,
request_spec,
filter_props,
volume,
snapshot_id,
image_id,
backup_id)
except Exception:
msg = _("Failed to create scheduler manager volume flow")
LOG.exception(msg)
raise exception.CinderException(msg)
with flow_utils.DynamicLogListener(flow_engine, logger=LOG):
flow_engine.run()
스케줄러의 워크플로우는 cinder/scheduler/flows/create_volume.py에 정의되어 있으며:
def get_flow(context, driver_api, request_spec=None,
filter_properties=None, volume=None, snapshot_id=None,
image_id=None, backup_id=None):
flow_params = {
'context': context,
'raw_request_spec': request_spec,
'filter_properties': filter_properties,
'volume': volume,
'snapshot_id': snapshot_id,
'image_id': image_id,
'backup_id': backup_id,
}
flow_name = ACTION.replace(":", "_") + "_scheduler"
scheduler_flow = linear_flow.Flow(flow_name)
scheduler_flow.add(ExtractSchedulerSpecTask(
rebind={'request_spec': 'raw_request_spec'}))
scheduler_flow.add(ScheduleCreateVolumeTask(driver_api))
return taskflow.engines.load(scheduler_flow, store=flow_params)
ScheduleCreateVolumeTask의 execute 메소드는 드라이버를 통해 볼륨 생성을 스케줄링합니다:
def execute(self, context, request_spec, filter_properties, volume):
try:
self.driver_api.schedule_create_volume(context, request_spec,
filter_properties)
except Exception as e:
self.message_api.create(
context,
message_field.Action.SCHEDULE_ALLOCATE_VOLUME,
resource_uuid=request_spec['volume_id'],
exception=e)
with excutils.save_and_reraise_exception(
reraise=not isinstance(e, exception.NoValidBackend)):
try:
self._handle_failure(context, request_spec, e)
finally:
common.error_out(volume, reason=e)
필터 스케줄러(cinder/scheduler/filter_scheduler.py)는 적절한 백엔드를 선택하고 볼륨 생성 요청을 볼륨 서비스로 전달합니다:
def schedule_create_volume(self, context, request_spec, filter_properties):
backend = self._schedule(context, request_spec, filter_properties)
if not backend:
raise exception.NoValidBackend(reason=_("No weighed backends available"))
backend_obj = backend.obj
volume_id = request_spec['volume_id']
updated_volume = driver.volume_update_db(
context, volume_id,
backend_obj.host,
backend_obj.cluster_name,
availability_zone=backend_obj.service['availability_zone'])
self._post_select_populate_filter_properties(filter_properties, backend_obj)
filter_properties.pop('context', None)
self.volume_rpcapi.create_volume(context, updated_volume, request_spec,
filter_properties, allow_reschedule=True)
볼륨 서비스(cinder/volume/manager.py)는 최종적으로 볼륨을 생성합니다:
@objects.Volume.set_workers
def create_volume(self, context, volume, request_spec=None,
filter_properties=None, allow_reschedule=True):
utils.log_unsupported_driver_warning(self.driver)
elevated_ctx = context.elevated()
if filter_properties is None:
filter_properties = {}
if request_spec is None:
request_spec = objects.RequestSpec()
try:
flow_engine = create_volume.get_flow(
elevated_ctx,
self,
self.db,
self.driver,
self.scheduler_rpcapi,
self.host,
volume,
allow_reschedule,
context,
request_spec,
filter_properties,
image_volume_cache=self.image_volume_cache,
)
except Exception:
msg = _("Create manager volume flow failed.")
LOG.exception(msg, resource={'type': 'volume', 'id': volume.id})
raise exception.CinderException(msg)
def _run_flow():
with flow_utils.DynamicLogListener(flow_engine, logger=LOG):
flow_engine.run()
locked_action = self._get_locked_action(request_spec)
try:
if locked_action is None:
_run_flow()
else:
with coordination.COORDINATOR.get_lock(locked_action):
_run_flow()
finally:
self._handle_reschedule_cleanup(flow_engine, volume)
volume.shared_targets = (
self.driver.capabilities.get('storage_protocol') == 'iSCSI' and
self.driver.capabilities.get('shared_targets', True))
volume.service_uuid = self.service_uuid
volume.save()
LOG.info("Created volume successfully.", resource=volume)
return volume.id
볼륨 매니저의 워크플로우는 다음과 같은 태스크들로 구성됩니다:
def get_flow(context, manager, db, driver, scheduler_rpcapi, host, volume,
allow_reschedule, reschedule_context, request_spec,
filter_properties, image_volume_cache=None):
flow_name = ACTION.replace(":", "_") + "_manager"
volume_flow = linear_flow.Flow(flow_name)
flow_params = {
'context': context,
'filter_properties': filter_properties,
'request_spec': request_spec,
'volume': volume,
}
volume_flow.add(ExtractVolumeRefTask(db, host, set_error=False))
retry = filter_properties.get('retry', None)
do_reschedule = allow_reschedule and request_spec and retry
volume_flow.add(OnFailureRescheduleTask(reschedule_context, db, driver,
scheduler_rpcapi, do_reschedule))
volume_flow.add(ExtractVolumeSpecTask(db),
NotifyVolumeActionTask(db, "create.start"),
CreateVolumeFromSpecTask(manager, db, driver,
image_volume_cache),
CreateVolumeOnFinishTask(db, "create.end"))
return taskflow.engines.load(volume_flow, store=flow_params)
CreateVolumeFromSpecTask는 볼륨 생성 타입에 따라 적절한 메소드를 호출합니다:
if create_type == 'raw':
model_update = self._create_raw_volume(volume, **volume_spec)
elif create_type == 'snap':
model_update = self._create_from_snapshot(context, volume, **volume_spec)
elif create_type == 'source_vol':
model_update = self._create_from_source_volume(context, volume, **volume_spec)
elif create_type == 'image':
model_update = self._create_from_image(context, volume, **volume_spec)
elif create_type == 'backup':
model_update, need_update_volume = self._create_from_backup(
context, volume, **volume_spec)
volume_spec.update({'need_update_volume': need_update_volume})
else:
raise exception.VolumeTypeNotFound(volume_type_id=create_type)
RAW 볼륨 생성의 경우:
def _create_raw_volume(self, volume, **kwargs):
try:
result = self.driver.create_volume(volume)
finally:
self._cleanup_cg_in_volume(volume)
return result
LVM 드라이버의 경우 실제 볼륨 생성은 다음과 같이 이루어집니다:
def create_volume(self, volume):
mirror_count = 0
if self.configuration.lvm_mirrors:
mirror_count = self.configuration.lvm_mirrors
self._create_volume(volume['name'],
self._sizestr(volume['size']),
self.configuration.lvm_type,
mirror_count)
def _create_volume(self, name, size, lvm_type, mirror_count, vg=None):
vg_ref = self.vg
if vg is not None:
vg_ref = vg
vg_ref.create_volume(name, size, lvm_type, mirror_count)
최종적으로 LVM 명령어를 통해 볼륨이 생성됩니다:
def create_volume(self, name, size_str, lv_type='default', mirror_count=0):
"""객체의 VG에 논리 볼륨을 생성합니다."""
if lv_type == 'thin':
pool_path = '%s/%s' % (self.vg_name, self.vg_thin_pool)
cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-T', '-V', size_str, '-n',
name, pool_path]
else:
cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-n', name, self.vg_name,
'-L', size_str]
if mirror_count > 0:
cmd.extend(['--type=mirror', '-m', mirror_count, '--nosync',
'--mirrorlog', 'mirrored'])
terras = int(size_str[:-1]) / 1024.0
if terras >= 1.5:
rsize = int(2 ** math.ceil(math.log(terras) / math.log(2)))
cmd.extend(['-R', str(rsize)])
try:
self._execute(*cmd,
root_helper=self._root_helper,
run_as_root=True)
except putils.ProcessExecutionError as err:
LOG.exception('Error creating Volume')
LOG.error('Cmd :%s', err.cmd)
LOG.error('StdOut :%s', err.stdout)
LOG.error('StdErr :%s', err.stderr)
LOG.error('Current state: %s',
self.get_all_volume_groups(self._root_helper))
raise