o ':j^@sddlZddlmZddlmmZzddlmZWney*ddlmZYnwddlTGdddej Z Gddde Z Gd d d e Z dS) N)Queue)*cs:eZdZdZdfdd Zd d Zd d Zd dZZS)ABNzrActivated Batch Normalization This gathers a `BatchNorm2d` and an activation function in a single module h㈵>皙?T leaky_relu{Gz?cstt|||_||_||_||_||_||_|jr/t t ||_ t t ||_n |dd|dd|dt ||dt ||dS)aCreates an Activated Batch Normalization module Parameters ---------- num_features : int Number of feature channels in the input and output. eps : float Small constant to prevent numerical issues. momentum : float Momentum factor applied to compute running statistics as. affine : bool If `True` apply learned scale and shift transformation after normalization. activation : str Name of the activation functions, one of: `leaky_relu`, `elu` or `none`. slope : float Negative slope for the `leaky_relu` activation. weightNbias running_mean running_var)superr__init__ num_featuresaffineepsmomentum activationslopenn Parametertorchonesr zerosr register_parameterregister_bufferreset_parametersselfrrrrrr __class__,/home/ubuntu/htdocs/OOTD3/SCHP/modules/bn.pyrs   z ABN.__init__cCsNtj|jdtj|jd|jr%tj|jdtj|jddSdS)Nrr)rinit constant_r r rr r )rr"r"r#r6s zABN.reset_parametersc Csvt||j|j|j|j|j|j|j}|j t krtj |ddS|j t kr-tj ||jddS|j tkr9tj|ddS|S)NT)inplace)negative_sloper&) functional batch_normr r r r trainingrrrACT_RELUreluACT_LEAKY_RELUrrACT_ELUelu)rxr"r"r#forward=s    z ABN.forwardcC:d}|jdkr |d7}n|d7}|jdd|jji|jSNz_{name}({num_features}, eps={eps}, momentum={momentum}, affine={affine}, activation={activation}rz, slope={slope}))namer"rformatr!__name____dict__rrepr"r"r#__repr__J   z ABN.__repr__rrTrr ) r8 __module__ __qualname____doc__rrr1r< __classcell__r"r"r r#r s # rcs*eZdZdZd fdd Zd d ZZS) InPlaceABNz%InPlace Activated Batch NormalizationrrTrr cstt|||||||dS)aCreates an InPlace Activated Batch Normalization module Parameters ---------- num_features : int Number of feature channels in the input and output. eps : float Small constant to prevent numerical issues. momentum : float Momentum factor applied to compute running statistics as. affine : bool If `True` apply learned scale and shift transformation after normalization. activation : str Name of the activation functions, one of: `leaky_relu`, `elu` or `none`. slope : float Negative slope for the `leaky_relu` activation. N)rrCrrr r"r#rWszInPlaceABN.__init__c C6t||j|j|j|j|j|j|j|j|j \}}}|SN) inplace_abnr r r r r*rrrrrr0_r"r"r#r1k zInPlaceABN.forwardr>)r8r?r@rArr1rBr"r"r r#rCTsrCc@s eZdZdZddZddZdS)InPlaceABNSynczInPlace Activated Batch Normalization with cross-GPU synchronization This assumes that it will be replicated across GPUs using the same mechanism as in `nn.DistributedDataParallel`. c CrDrE) inplace_abn_syncr r r r r*rrrrrGr"r"r#r1vrIzInPlaceABNSync.forwardcCr2r3r6r:r"r"r#r<{r=zInPlaceABNSync.__repr__N)r8r?r@rAr1r<r"r"r"r#rJqs rJ) rtorch.nnrtorch.nn.functionalr(queuer ImportError functionsModulerrCrJr"r"r"r#s  G