libMesh
Namespaces | Classes | Typedefs | Functions | Variables
libMesh::Parallel Namespace Reference

The Parallel namespace is for wrapper functions for common general parallel synchronization tasks. More...

Namespaces

 Utils
 

Classes

struct  Attributes
 
struct  Attributes< unsigned long long >
 
class  BinSorter
 Perform a parallel sort using a bin-sort method. More...
 
class  Communicator
 Encapsulates the MPI_Comm object. More...
 
struct  data_type
 
class  DataPlusInt
 Types combined with an int. More...
 
class  DataType
 Encapsulates the MPI_Datatype. More...
 
struct  dependent_false
 
class  FakeCommunicator
 
struct  false_type
 
class  Histogram
 Defines a histogram to be used in parallel in conjunction with a BinSorter. More...
 
class  MessageTag
 Encapsulates the MPI tag integers. More...
 
class  OpFunction
 Templated class to provide the appropriate MPI reduction operations for use with built-in C types or simple C++ constructions. More...
 
class  OpFunction< Point >
 
class  OpFunction< TypeVector< T > >
 
class  OpFunction< VectorValue< T > >
 
class  Packing
 Define data types and (un)serialization functions for use when encoding a potentially-variable-size object of type T. More...
 
class  Packing< const Elem * >
 
class  Packing< const Node * >
 
class  Packing< Elem * >
 
class  Packing< Node * >
 
class  Packing< std::basic_string< T > >
 
struct  PostWaitCopyBuffer
 
struct  PostWaitDeleteBuffer
 
struct  PostWaitUnpackBuffer
 
struct  PostWaitWork
 A class that can be subclassed to allow other code to perform work after a MPI_Wait succeeds. More...
 
struct  request
 
class  Request
 Encapsulates the MPI_Request. More...
 
class  Sort
 The parallel sorting method is templated on the type of data which is to be sorted. More...
 
class  StandardType
 Templated class to provide the appropriate MPI datatype for use with built-in C types or simple C++ constructions. More...
 
class  StandardType< Hilbert::HilbertIndices >
 
class  StandardType< Point >
 
class  StandardType< std::complex< T > >
 
class  StandardType< std::pair< T1, T2 > >
 
class  StandardType< TensorValue< T > >
 
class  StandardType< TypeTensor< T > >
 
class  StandardType< TypeVector< T > >
 
class  StandardType< unsigned long long >
 
class  StandardType< VectorValue< T > >
 
struct  status
 
class  Status
 Encapsulates the MPI_Status struct. More...
 
struct  SyncEverything
 
class  TypeVectorOpFunction
 

Typedefs

typedef MPI_Datatype data_type
 Data types for communication. More...
 
typedef MPI_Request request
 Request object for non-blocking I/O. More...
 
typedef MPI_Status status
 Status object for querying messages. More...
 
typedef MPI_Comm communicator
 Communicator object for talking with subsets of processors. More...
 
typedef std::pair< Hilbert::HilbertIndices, unique_id_typeDofObjectKey
 

Functions

template<typename T >
data_type dataplusint_type ()
 Templated function to return the appropriate MPI datatype for use with built-in C types when combined with an int. More...
 
Status wait (Request &r)
 Wait for a non-blocking send or receive to finish. More...
 
void wait (std::vector< Request > &r)
 Wait for a non-blocking send or receive to finish. More...
 
template<typename Context , typename buffertype , typename OutputIter , typename T >
void unpack_range (const typename std::vector< buffertype > &buffer, Context *context, OutputIter out, const T *output_type)
 Decode a range of potentially-variable-size objects from a data array. More...
 
template<typename Context , typename buffertype , typename Iter >
Iter pack_range (const Context *context, Iter range_begin, const Iter range_end, typename std::vector< buffertype > &buffer, std::size_t approx_buffer_size=1000000)
 Encode a range of potentially-variable-size objects to a data array. More...
 
template<typename Context , typename Iter >
std::size_t packed_range_size (const Context *context, Iter range_begin, const Iter range_end)
 Return the total buffer size needed to encode a range of potentially-variable-size objects to a data array. More...
 
template<typename Iterator , typename DofObjType , typename SyncFunctor >
void sync_dofobject_data_by_xyz (const Communicator &comm, const Iterator &range_begin, const Iterator &range_end, LocationMap< DofObjType > *location_map, SyncFunctor &sync)
 Request data about a range of ghost nodes uniquely identified by their xyz location or a range of active ghost elements uniquely identified by their centroids' xyz location. More...
 
template<typename Iterator , typename SyncFunctor >
void sync_dofobject_data_by_id (const Communicator &comm, const Iterator &range_begin, const Iterator &range_end, SyncFunctor &sync)
 Request data about a range of ghost dofobjects uniquely identified by their id. More...
 
template<typename Iterator , typename DofObjectCheckFunctor , typename SyncFunctor >
void sync_dofobject_data_by_id (const Communicator &comm, const Iterator &range_begin, const Iterator &range_end, const DofObjectCheckFunctor &dofobj_check, SyncFunctor &sync)
 Request data about a range of ghost dofobjects uniquely identified by their id. More...
 
template<typename Iterator , typename SyncFunctor >
void sync_element_data_by_parent_id (MeshBase &mesh, const Iterator &range_begin, const Iterator &range_end, SyncFunctor &sync)
 Request data about a range of ghost elements uniquely identified by their parent id and which child they are. More...
 
template<typename ElemCheckFunctor , typename NodeCheckFunctor , typename SyncFunctor >
void sync_node_data_by_element_id (MeshBase &mesh, const MeshBase::const_element_iterator &range_begin, const MeshBase::const_element_iterator &range_end, const ElemCheckFunctor &elem_check, const NodeCheckFunctor &node_check, SyncFunctor &sync)
 Request data about a range of ghost nodes uniquely identified by an element id and local node id. More...
 
template<typename Iterator , typename DofObjType , typename SyncFunctor >
void sync_dofobject_data_by_xyz (const Communicator &comm, const Iterator &range_begin, const Iterator &range_end, LocationMap< DofObjType > &location_map, SyncFunctor &sync)
 
 LIBMESH_INT_TYPE (char, MPI_CHAR)
 
 LIBMESH_INT_TYPE (signed char, MPI_SIGNED_CHAR)
 
 LIBMESH_INT_TYPE (unsigned char, MPI_UNSIGNED_CHAR)
 
 LIBMESH_INT_TYPE (short int, MPI_SHORT)
 
 LIBMESH_INT_TYPE (unsigned short int, MPI_UNSIGNED_SHORT)
 
 LIBMESH_INT_TYPE (int, MPI_INT)
 
 LIBMESH_INT_TYPE (unsigned int, MPI_UNSIGNED)
 
 LIBMESH_INT_TYPE (long, MPI_LONG)
 
 LIBMESH_INT_TYPE (long long, MPI_LONG_LONG_INT)
 
 LIBMESH_INT_TYPE (unsigned long, MPI_UNSIGNED_LONG)
 
 LIBMESH_INT_TYPE (unsigned long long, MPI_UNSIGNED_LONG_LONG)
 
 LIBMESH_PARALLEL_INTEGER_OPS (unsigned long long)
 
 LIBMESH_FLOAT_TYPE (float, MPI_FLOAT)
 
 LIBMESH_FLOAT_TYPE (double, MPI_DOUBLE)
 
 LIBMESH_FLOAT_TYPE (long double, MPI_LONG_DOUBLE)
 
 LIBMESH_CONTAINER_TYPE (std::set)
 
 LIBMESH_CONTAINER_TYPE (std::vector)
 
template<typename Context , typename buffertype , typename OutputIter , typename T >
void unpack_range (const std::vector< buffertype > &buffer, Context *context, OutputIter out_iter, const T *)
 Helper function for range unpacking. More...
 
void barrier (const Communicator &comm=Communicator_World)
 
template<typename T >
bool verify (const T &r, const Communicator &comm=Communicator_World)
 
template<typename T >
void min (T &r, const Communicator &comm=Communicator_World)
 
template<typename T , typename U >
void minloc (T &r, U &min_id, const Communicator &comm=Communicator_World)
 
template<typename T >
void max (T &r, const Communicator &comm=Communicator_World)
 
template<typename T , typename U >
void maxloc (T &r, U &max_id, const Communicator &comm=Communicator_World)
 
template<typename T >
void sum (T &r, const Communicator &comm=Communicator_World)
 
template<typename T >
void set_union (T &data, const unsigned int root_id, const Communicator &comm=Communicator_World)
 
template<typename T >
void set_union (T &data, const Communicator &comm=Communicator_World)
 
status probe (const unsigned int src_processor_id, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void send (const unsigned int dest_processor_id, const T &data, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void send (const unsigned int dest_processor_id, const T &data, Request &req, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void send (const unsigned int dest_processor_id, const T &data, const DataType &type, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void send (const unsigned int dest_processor_id, const T &data, const DataType &type, Request &req, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename Context , typename Iter >
void send_packed_range (const unsigned int dest_processor_id, const Context *context, Iter range_begin, const Iter range_end, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename Context , typename Iter >
void send_packed_range (const unsigned int dest_processor_id, const Context *context, Iter range_begin, const Iter range_end, Request &req, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void nonblocking_send (const unsigned int dest_processor_id, T &buf, const DataType &type, Request &r, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void nonblocking_send (const unsigned int dest_processor_id, T &buf, Request &r, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
Status receive (const unsigned int src_processor_id, T &buf, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void receive (const unsigned int src_processor_id, T &buf, Request &req, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
Status receive (const unsigned int src_processor_id, T &buf, const DataType &type, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void receive (const unsigned int src_processor_id, T &buf, const DataType &type, Request &req, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename Context , typename OutputIter , typename T >
void receive_packed_range (const unsigned int src_processor_id, Context *context, OutputIter out_iter, const T *output_type, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void nonblocking_receive (const unsigned int src_processor_id, T &buf, const DataType &type, Request &r, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void nonblocking_receive (const unsigned int src_processor_id, T &buf, Request &r, const MessageTag &tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T1 , typename T2 >
void send_receive (const unsigned int dest_processor_id, T1 &send, const unsigned int source_processor_id, T2 &recv, const MessageTag &send_tag=no_tag, const MessageTag &recv_tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename Context1 , typename RangeIter , typename Context2 , typename OutputIter , typename T >
void send_receive_packed_range (const unsigned int dest_processor_id, const Context1 *context1, RangeIter send_begin, const RangeIter send_end, const unsigned int source_processor_id, Context2 *context2, OutputIter out_iter, const T *output_type, const MessageTag &send_tag=no_tag, const MessageTag &recv_tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T1 , typename T2 >
void send_receive (const unsigned int dest_processor_id, T1 &send, const DataType &type1, const unsigned int source_processor_id, T2 &recv, const DataType &type2, const MessageTag &send_tag=no_tag, const MessageTag &recv_tag=any_tag, const Communicator &comm=Communicator_World)
 
template<typename T >
void gather (const unsigned int root_id, T send, std::vector< T > &recv, const Communicator &comm=Communicator_World)
 
template<typename T >
void gather (const unsigned int root_id, std::vector< T > &r, const Communicator &comm=Communicator_World)
 
template<typename T >
void allgather (T send, std::vector< T > &recv, const Communicator &comm=Communicator_World)
 
template<typename T >
void allgather (std::vector< T > &r, const bool identical_buffer_sizes=false, const Communicator &comm=Communicator_World)
 
template<typename Context , typename Iter , typename OutputIter >
void gather_packed_range (const unsigned int root_id, Context *context, Iter range_begin, const Iter range_end, OutputIter out_iter, const Communicator &comm=Communicator_World)
 
template<typename Context , typename Iter , typename OutputIter >
void allgather_packed_range (Context *context, Iter range_begin, const Iter range_end, OutputIter out_iter, const Communicator &comm=Communicator_World)
 
template<typename T >
void alltoall (std::vector< T > &r, const Communicator &comm=Communicator_World)
 
template<typename T >
void broadcast (T &data, const unsigned int root_id=0, const Communicator &comm=Communicator_World)
 
template<typename Context , typename OutputContext , typename Iter , typename OutputIter >
void broadcast_packed_range (const Context *context1, Iter range_begin, const Iter range_end, OutputContext *context2, OutputIter out_iter, const unsigned int root_id=0, const Communicator &comm=Communicator_World)
 
template<>
data_type dataplusint_type< short int > ()
 
template<>
data_type dataplusint_type< int > ()
 
template<>
data_type dataplusint_type< long > ()
 
template<>
data_type dataplusint_type< float > ()
 
template<>
data_type dataplusint_type< double > ()
 
template<>
data_type dataplusint_type< long double > ()
 

Variables

const unsigned int any_source
 Accept from any source. More...
 
const MessageTag any_tag = MessageTag(MPI_ANY_TAG)
 Default message tag ids. More...
 
const MessageTag no_tag = MessageTag(0)
 
FakeCommunicatorCommunicator_World = CommWorld
 

Detailed Description

The Parallel namespace is for wrapper functions for common general parallel synchronization tasks.

For MPI 1.1 compatibility, temporary buffers are used instead of MPI 2's MPI_IN_PLACE

Typedef Documentation

Communicator object for talking with subsets of processors.

Definition at line 181 of file parallel.h.

typedef MPI_Datatype libMesh::Parallel::data_type

Data types for communication.

Definition at line 166 of file parallel.h.

typedef Hilbert::HilbertIndices libMesh::Parallel::DofObjectKey

Definition at line 70 of file parallel_hilbert.h.

typedef MPI_Request libMesh::Parallel::request

Request object for non-blocking I/O.

Definition at line 171 of file parallel.h.

typedef MPI_Status libMesh::Parallel::status

Status object for querying messages.

Definition at line 176 of file parallel.h.

Function Documentation

template<typename T >
void libMesh::Parallel::allgather ( send,
std::vector< T > &  recv,
const Communicator comm = Communicator_World 
)

Definition at line 1276 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::allgather(), libMesh::Parallel::Communicator::allgather_packed_range(), libMesh::Parallel::Communicator::gather(), libMesh::Parallel::Communicator::send_mode(), and libMesh::Parallel::Communicator::set_union().

1279 { comm.allgather(send, recv); }
void send(const unsigned int dest_processor_id, const T &data, const DataType &type, Request &req, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
template<typename T >
void libMesh::Parallel::allgather ( std::vector< T > &  r,
const bool  identical_buffer_sizes = false,
const Communicator comm = Communicator_World 
)

Definition at line 1282 of file parallel_implementation.h.

1285 { comm.allgather(r, identical_buffer_sizes); }
template<typename Context , typename Iter , typename OutputIter >
void libMesh::Parallel::allgather_packed_range ( Context *  context,
Iter  range_begin,
const Iter  range_end,
OutputIter  out_iter,
const Communicator comm = Communicator_World 
)

Definition at line 1297 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1302 { comm.allgather_packed_range(context, range_begin, range_end, out_iter); }
template<typename T >
void libMesh::Parallel::alltoall ( std::vector< T > &  r,
const Communicator comm = Communicator_World 
)

Definition at line 1305 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1307 { comm.alltoall(r); }
void libMesh::Parallel::barrier ( const Communicator comm = Communicator_World)

Definition at line 1024 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1025 {
1026  comm.barrier();
1027 }
template<typename T >
void libMesh::Parallel::broadcast ( T &  data,
const unsigned int  root_id = 0,
const Communicator comm = Communicator_World 
)

Definition at line 1310 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::broadcast(), libMesh::Parallel::Communicator::broadcast_packed_range(), libMesh::Parallel::Communicator::scatter(), and libMesh::Parallel::Communicator::send_mode().

1313 { comm.broadcast(data, root_id); }
IterBase * data
Ideally this private member data should have protected access.
template<typename Context , typename OutputContext , typename Iter , typename OutputIter >
void libMesh::Parallel::broadcast_packed_range ( const Context *  context1,
Iter  range_begin,
const Iter  range_end,
OutputContext *  context2,
OutputIter  out_iter,
const unsigned int  root_id = 0,
const Communicator comm = Communicator_World 
)

Definition at line 1316 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1323 { comm.broadcast_packed_range(context1, range_begin, range_end, context2, out_iter, root_id); }
template<typename T >
data_type libMesh::Parallel::dataplusint_type ( )

Templated function to return the appropriate MPI datatype for use with built-in C types when combined with an int.

Definition at line 1409 of file parallel_implementation.h.

1409 { return MPI_DOUBLE_INT; }

Definition at line 1406 of file parallel_implementation.h.

1406 { return MPI_FLOAT_INT; }

Definition at line 1400 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::maxloc(), and libMesh::Parallel::Communicator::minloc().

1400 { return MPI_2INT; }

Definition at line 1403 of file parallel_implementation.h.

1403 { return MPI_LONG_INT; }
template<>
data_type libMesh::Parallel::dataplusint_type< long double > ( )

Definition at line 1412 of file parallel_implementation.h.

1412 { return MPI_LONG_DOUBLE_INT; }

Definition at line 1397 of file parallel_implementation.h.

1397 { return MPI_SHORT_INT; }
template<typename T >
void libMesh::Parallel::gather ( const unsigned int  root_id,
send,
std::vector< T > &  recv,
const Communicator comm = Communicator_World 
)

Definition at line 1263 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::gather(), libMesh::Parallel::Communicator::gather_packed_range(), libMesh::Parallel::Communicator::send_mode(), and libMesh::Parallel::Communicator::set_union().

1267 { comm.gather(root_id, send, recv); }
void send(const unsigned int dest_processor_id, const T &data, const DataType &type, Request &req, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
template<typename T >
void libMesh::Parallel::gather ( const unsigned int  root_id,
std::vector< T > &  r,
const Communicator comm = Communicator_World 
)

Definition at line 1270 of file parallel_implementation.h.

1273 { comm.gather(root_id, r); }
template<typename Context , typename Iter , typename OutputIter >
void libMesh::Parallel::gather_packed_range ( const unsigned int  root_id,
Context *  context,
Iter  range_begin,
const Iter  range_end,
OutputIter  out_iter,
const Communicator comm = Communicator_World 
)

Definition at line 1288 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1294 { comm.gather_packed_range(root_id, context, range_begin, range_end, out_iter); }
libMesh::Parallel::LIBMESH_CONTAINER_TYPE ( std::set  )
libMesh::Parallel::LIBMESH_CONTAINER_TYPE ( std::vector  )
libMesh::Parallel::LIBMESH_FLOAT_TYPE ( float  ,
MPI_FLOAT   
)
libMesh::Parallel::LIBMESH_FLOAT_TYPE ( double  ,
MPI_DOUBLE   
)
libMesh::Parallel::LIBMESH_FLOAT_TYPE ( long  double,
MPI_LONG_DOUBLE   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( char  ,
MPI_CHAR   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( signed  char,
MPI_SIGNED_CHAR   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( unsigned  char,
MPI_UNSIGNED_CHAR   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( short  int,
MPI_SHORT   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( unsigned short  int,
MPI_UNSIGNED_SHORT   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( int  ,
MPI_INT   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( unsigned  int,
MPI_UNSIGNED   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( long  ,
MPI_LONG   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( long  long,
MPI_LONG_LONG_INT   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( unsigned  long,
MPI_UNSIGNED_LONG   
)
libMesh::Parallel::LIBMESH_INT_TYPE ( unsigned long  long,
MPI_UNSIGNED_LONG_LONG   
)
libMesh::Parallel::LIBMESH_PARALLEL_INTEGER_OPS ( unsigned long  long)
template<typename T >
void libMesh::Parallel::max ( T &  r,
const Communicator comm = Communicator_World 
)
template<typename T , typename U >
void libMesh::Parallel::maxloc ( T &  r,
U &  max_id,
const Communicator comm = Communicator_World 
)

Definition at line 1051 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1054 { comm.maxloc(r, max_id); }
template<typename T >
void libMesh::Parallel::min ( T &  r,
const Communicator comm = Communicator_World 
)
template<typename T , typename U >
void libMesh::Parallel::minloc ( T &  r,
U &  min_id,
const Communicator comm = Communicator_World 
)

Definition at line 1040 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1043 { comm.minloc(r, min_id); }
template<typename T >
void libMesh::Parallel::nonblocking_receive ( const unsigned int  src_processor_id,
T &  buf,
const DataType type,
Request r,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1201 of file parallel_implementation.h.

1207 { comm.receive (src_processor_id, buf, type, r, tag); }
template<typename T >
void libMesh::Parallel::nonblocking_receive ( const unsigned int  src_processor_id,
T &  buf,
Request r,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1210 of file parallel_implementation.h.

1215 { comm.receive (src_processor_id, buf, r, tag); }
template<typename T >
void libMesh::Parallel::nonblocking_send ( const unsigned int  dest_processor_id,
T &  buf,
const DataType type,
Request r,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1131 of file parallel_implementation.h.

1137 { comm.send (dest_processor_id, buf, type, r, tag); }
template<typename T >
void libMesh::Parallel::nonblocking_send ( const unsigned int  dest_processor_id,
T &  buf,
Request r,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1140 of file parallel_implementation.h.

1145 { comm.send (dest_processor_id, buf, r, tag); }
template<typename Context , typename buffertype , typename Iter >
Iter libMesh::Parallel::pack_range ( const Context *  context,
Iter  range_begin,
const Iter  range_end,
typename std::vector< buffertype > &  buffer,
std::size_t  approx_buffer_size = 1000000 
)

Encode a range of potentially-variable-size objects to a data array.

Helper function for range packing.

The data will be buffered in vectors with lengths that do not exceed the sum of approx_buffer_size and the size of an individual packed object.

Definition at line 540 of file parallel_implementation.h.

References libMesh::Parallel::Packing< T >::pack(), libMesh::Parallel::Packing< T >::packable_size(), and libMesh::Parallel::Packing< T >::packed_size().

Referenced by libMesh::Parallel::Communicator::allgather_packed_range(), libMesh::Parallel::Communicator::broadcast_packed_range(), libMesh::Parallel::Communicator::gather_packed_range(), libMesh::Parallel::Communicator::nonblocking_send_packed_range(), libMesh::Parallel::Communicator::receive(), and libMesh::Parallel::Communicator::send_packed_range().

549 {
550  typedef typename std::iterator_traits<Iter>::value_type T;
551 
552  // Count the total size of and preallocate buffer for efficiency.
553  // Prepare to stop early if the buffer would be too large.
554  std::size_t buffer_size = 0;
555  Iter range_stop = range_begin;
556  for (; range_stop != range_end && buffer_size < approx_buffer_size;
557  ++range_stop)
558  {
559  std::size_t next_buffer_size =
560  Parallel::Packing<T>::packable_size(*range_stop, context);
561  buffer_size += next_buffer_size;
562  }
563  buffer.reserve(buffer.size() + buffer_size);
564 
565  // Pack the objects into the buffer
566  for (; range_begin != range_stop; ++range_begin)
567  {
568 #ifndef NDEBUG
569  std::size_t old_size = buffer.size();
570 #endif
571 
572  Parallel::Packing<T>::pack
573  (*range_begin, back_inserter(buffer), context);
574 
575 #ifndef NDEBUG
576  unsigned int my_packable_size =
577  Parallel::Packing<T>::packable_size(*range_begin, context);
578  unsigned int my_packed_size =
579  Parallel::Packing<T>::packed_size (buffer.begin() + old_size);
580  libmesh_assert_equal_to (my_packable_size, my_packed_size);
581  libmesh_assert_equal_to (buffer.size(), old_size + my_packable_size);
582 #endif
583  }
584 
585  return range_stop;
586 }
template<typename Context , typename Iter >
std::size_t libMesh::Parallel::packed_range_size ( const Context *  context,
Iter  range_begin,
const Iter  range_end 
)

Return the total buffer size needed to encode a range of potentially-variable-size objects to a data array.

Helper function for range packing.

Definition at line 519 of file parallel_implementation.h.

References libMesh::Parallel::Packing< T >::packable_size().

Referenced by libMesh::Parallel::Communicator::send_packed_range().

522 {
523  typedef typename std::iterator_traits<Iter>::value_type T;
524 
525  std::size_t buffer_size = 0;
526  for (Iter range_count = range_begin;
527  range_count != range_end;
528  ++range_count)
529  {
530  buffer_size += Parallel::Packing<T>::packable_size(*range_count, context);
531  }
532  return buffer_size;
533 }
status libMesh::Parallel::probe ( const unsigned int  src_processor_id,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)
template<typename T >
Status libMesh::Parallel::receive ( const unsigned int  src_processor_id,
T &  buf,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)
template<typename T >
void libMesh::Parallel::receive ( const unsigned int  src_processor_id,
T &  buf,
Request req,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1155 of file parallel_implementation.h.

1160 { comm.receive (src_processor_id, buf, req, tag); }
template<typename T >
Status libMesh::Parallel::receive ( const unsigned int  src_processor_id,
T &  buf,
const DataType type,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1163 of file parallel_implementation.h.

1168 { return comm.receive (src_processor_id, buf, type, tag); }
template<typename T >
void libMesh::Parallel::receive ( const unsigned int  src_processor_id,
T &  buf,
const DataType type,
Request req,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1171 of file parallel_implementation.h.

1177 { comm.receive (src_processor_id, buf, type, req, tag); }
template<typename Context , typename OutputIter , typename T >
void libMesh::Parallel::receive_packed_range ( const unsigned int  src_processor_id,
Context *  context,
OutputIter  out_iter,
const T *  output_type,
const MessageTag tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1180 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode(), and libMesh::Parallel::Communicator::send_receive_packed_range().

1186 {
1187  comm.receive_packed_range (src_processor_id, context, out_iter,
1188  output_type, tag);
1189 }
template<typename T >
void libMesh::Parallel::send ( const unsigned int  dest_processor_id,
const T &  data,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)
template<typename T >
void libMesh::Parallel::send ( const unsigned int  dest_processor_id,
const T &  data,
Request req,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1084 of file parallel_implementation.h.

1089 { comm.send(dest_processor_id, data, req, tag); }
IterBase * data
Ideally this private member data should have protected access.
template<typename T >
void libMesh::Parallel::send ( const unsigned int  dest_processor_id,
const T &  data,
const DataType type,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1092 of file parallel_implementation.h.

1097 { comm.send(dest_processor_id, data, type, tag); }
IterBase * data
Ideally this private member data should have protected access.
template<typename T >
void libMesh::Parallel::send ( const unsigned int  dest_processor_id,
const T &  data,
const DataType type,
Request req,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1100 of file parallel_implementation.h.

1106 { comm.send(dest_processor_id, data, type, req, tag); }
IterBase * data
Ideally this private member data should have protected access.
template<typename Context , typename Iter >
void libMesh::Parallel::send_packed_range ( const unsigned int  dest_processor_id,
const Context *  context,
Iter  range_begin,
const Iter  range_end,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1110 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode(), and libMesh::Parallel::Communicator::send_receive_packed_range().

1116 { comm.send_packed_range(dest_processor_id, context, range_begin, range_end, tag); }
template<typename Context , typename Iter >
void libMesh::Parallel::send_packed_range ( const unsigned int  dest_processor_id,
const Context *  context,
Iter  range_begin,
const Iter  range_end,
Request req,
const MessageTag tag = no_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1120 of file parallel_implementation.h.

1127 { comm.send_packed_range(dest_processor_id, context, range_begin, range_end, req, tag); }
template<typename T1 , typename T2 >
void libMesh::Parallel::send_receive ( const unsigned int  dest_processor_id,
T1 &  send,
const unsigned int  source_processor_id,
T2 &  recv,
const MessageTag send_tag = no_tag,
const MessageTag recv_tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1218 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode(), and libMesh::Parallel::Communicator::send_receive().

1225 { comm.send_receive(dest_processor_id, send, source_processor_id, recv,
1226  send_tag, recv_tag); }
void send(const unsigned int dest_processor_id, const T &data, const DataType &type, Request &req, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
template<typename T1 , typename T2 >
void libMesh::Parallel::send_receive ( const unsigned int  dest_processor_id,
T1 &  send,
const DataType type1,
const unsigned int  source_processor_id,
T2 &  recv,
const DataType type2,
const MessageTag send_tag = no_tag,
const MessageTag recv_tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1250 of file parallel_implementation.h.

1259 { comm.send_receive(dest_processor_id, send, type1, source_processor_id,
1260  recv, type2, send_tag, recv_tag); }
void send(const unsigned int dest_processor_id, const T &data, const DataType &type, Request &req, const MessageTag &tag=no_tag, const Communicator &comm=Communicator_World)
template<typename Context1 , typename RangeIter , typename Context2 , typename OutputIter , typename T >
void libMesh::Parallel::send_receive_packed_range ( const unsigned int  dest_processor_id,
const Context1 *  context1,
RangeIter  send_begin,
const RangeIter  send_end,
const unsigned int  source_processor_id,
Context2 *  context2,
OutputIter  out_iter,
const T *  output_type,
const MessageTag send_tag = no_tag,
const MessageTag recv_tag = any_tag,
const Communicator comm = Communicator_World 
)

Definition at line 1230 of file parallel_implementation.h.

Referenced by libMesh::Parallel::Communicator::send_mode().

1241 {
1242  comm.send_receive_packed_range(dest_processor_id, context1,
1243  send_begin, send_end,
1244  source_processor_id, context2,
1245  out_iter, output_type,
1246  send_tag, recv_tag);
1247 }
template<typename T >
void libMesh::Parallel::set_union ( T &  data,
const unsigned int  root_id,
const Communicator comm = Communicator_World 
)

Definition at line 1062 of file parallel_implementation.h.

Referenced by libMesh::Nemesis_IO_Helper::compute_border_node_ids(), and libMesh::Parallel::Communicator::send_mode().

1064 { comm.set_union(data, root_id); }
IterBase * data
Ideally this private member data should have protected access.
template<typename T >
void libMesh::Parallel::set_union ( T &  data,
const Communicator comm = Communicator_World 
)

Definition at line 1067 of file parallel_implementation.h.

1069 { comm.set_union(data); }
IterBase * data
Ideally this private member data should have protected access.
template<typename T >
void libMesh::Parallel::sum ( T &  r,
const Communicator comm = Communicator_World 
)
template<typename Iterator , typename SyncFunctor >
void libMesh::Parallel::sync_dofobject_data_by_id ( const Communicator comm,
const Iterator &  range_begin,
const Iterator &  range_end,
SyncFunctor &  sync 
)

Request data about a range of ghost dofobjects uniquely identified by their id.

Fulfill requests with sync.gather_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data), by resizing and setting the values of the data vector. Respond to fulfillment with sync.act_on_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data) The user must define Parallel::StandardType<sync::datum> if sync::datum isn't a built-in type.

Definition at line 300 of file parallel_ghost_sync.h.

References libMesh::Parallel::SyncEverything::SyncEverything().

Referenced by libMesh::MeshTools::correct_node_proc_ids(), libMesh::MeshCommunication::gather_neighboring_elements(), libMesh::MeshRefinement::make_coarsening_compatible(), libMesh::MeshCommunication::make_elems_parallel_consistent(), libMesh::MeshRefinement::make_flags_parallel_consistent(), libMesh::MeshCommunication::make_node_unique_ids_parallel_consistent(), libMesh::MeshCommunication::make_p_levels_parallel_consistent(), libMesh::FEMSystem::mesh_position_set(), libMesh::LaplaceMeshSmoother::smooth(), and libMesh::MeshRefinement::uniformly_coarsen().

304 {
305  sync_dofobject_data_by_id(comm, range_begin, range_end, SyncEverything(), sync);
306 }
void sync_dofobject_data_by_id(const Communicator &comm, const Iterator &range_begin, const Iterator &range_end, const DofObjectCheckFunctor &dofobj_check, SyncFunctor &sync)
Request data about a range of ghost dofobjects uniquely identified by their id.
template<typename Iterator , typename DofObjectCheckFunctor , typename SyncFunctor >
void libMesh::Parallel::sync_dofobject_data_by_id ( const Communicator comm,
const Iterator &  range_begin,
const Iterator &  range_end,
const DofObjectCheckFunctor &  dofobj_check,
SyncFunctor &  sync 
)

Request data about a range of ghost dofobjects uniquely identified by their id.

Elements within the range can be excluded from the request by returning false from dofobj_check(dof_object)

Definition at line 311 of file parallel_ghost_sync.h.

References data, libMesh::DofObject::id(), libMesh::DofObject::invalid_processor_id, libMesh::libmesh_assert(), libMesh::DofObject::processor_id(), libMesh::Parallel::Communicator::rank(), libMesh::Parallel::Communicator::send_receive(), and libMesh::Parallel::Communicator::size().

316 {
317  // This function must be run on all processors at once
318  libmesh_parallel_only(comm);
319 
320  // Count the objects to ask each processor about
321  std::vector<dof_id_type>
322  ghost_objects_from_proc(comm.size(), 0);
323 
324  for (Iterator it = range_begin; it != range_end; ++it)
325  {
326  DofObject * obj = *it;
327  libmesh_assert (obj);
328 
329  // We may want to pass Elem* or Node* to the check function, not
330  // just DofObject*
331  if (!dofobj_check(*it))
332  continue;
333 
334  processor_id_type obj_procid = obj->processor_id();
335  if (obj_procid != DofObject::invalid_processor_id)
336  ghost_objects_from_proc[obj_procid]++;
337  }
338 
339  // Request sets to send to each processor
340  std::vector<std::vector<dof_id_type>>
341  requested_objs_id(comm.size());
342 
343  // We know how many objects live on each processor, so reserve()
344  // space for each.
345  for (processor_id_type p=0; p != comm.size(); ++p)
346  if (p != comm.rank())
347  {
348  requested_objs_id[p].reserve(ghost_objects_from_proc[p]);
349  }
350  for (Iterator it = range_begin; it != range_end; ++it)
351  {
352  DofObject * obj = *it;
353 
354  if (!dofobj_check(*it))
355  continue;
356 
357  processor_id_type obj_procid = obj->processor_id();
358  if (obj_procid == comm.rank() ||
359  obj_procid == DofObject::invalid_processor_id)
360  continue;
361 
362  requested_objs_id[obj_procid].push_back(obj->id());
363  }
364 
365  // Trade requests with other processors
366  for (processor_id_type p=1; p != comm.size(); ++p)
367  {
368  // Trade my requests with processor procup and procdown
369  const processor_id_type procup =
370  cast_int<processor_id_type>
371  ((comm.rank() + p) % comm.size());
372  const processor_id_type procdown =
373  cast_int<processor_id_type>
374  ((comm.size() + comm.rank() - p) %
375  comm.size());
376  std::vector<dof_id_type> request_to_fill_id;
377  comm.send_receive(procup, requested_objs_id[procup],
378  procdown, request_to_fill_id);
379 
380  // Gather whatever data the user wants
381  std::vector<typename SyncFunctor::datum> data;
382  sync.gather_data(request_to_fill_id, data);
383 
384  // Trade back the results
385  std::vector<typename SyncFunctor::datum> received_data;
386  comm.send_receive(procdown, data,
387  procup, received_data);
388  libmesh_assert_equal_to (requested_objs_id[procup].size(),
389  received_data.size());
390 
391  // Let the user process the results
392  sync.act_on_data(requested_objs_id[procup], received_data);
393  }
394 }
uint8_t processor_id_type
Definition: id_types.h:99
libmesh_assert(j)
IterBase * data
Ideally this private member data should have protected access.
template<typename Iterator , typename DofObjType , typename SyncFunctor >
void libMesh::Parallel::sync_dofobject_data_by_xyz ( const Communicator comm,
const Iterator &  range_begin,
const Iterator &  range_end,
LocationMap< DofObjType > *  location_map,
SyncFunctor &  sync 
)

Request data about a range of ghost nodes uniquely identified by their xyz location or a range of active ghost elements uniquely identified by their centroids' xyz location.

Fulfill requests with sync.gather_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data), by resizing and setting the values of the data vector. Respond to fulfillment with sync.act_on_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data) The user must define Parallel::StandardType<sync::datum> if sync::datum isn't a built-in type. The user-provided location_map will be used and left unchanged if it is provided, or filled and cleared if it is empty.

template<typename Iterator , typename DofObjType , typename SyncFunctor >
void libMesh::Parallel::sync_dofobject_data_by_xyz ( const Communicator comm,
const Iterator &  range_begin,
const Iterator &  range_end,
LocationMap< DofObjType > &  location_map,
SyncFunctor &  sync 
)

Definition at line 177 of file parallel_ghost_sync.h.

References data, libMesh::LocationMap< T >::empty(), libMesh::LocationMap< T >::find(), libMesh::DofObject::invalid_processor_id, libMesh::libmesh_assert(), libMesh::Parallel::Communicator::max(), libMesh::LocationMap< T >::point_of(), libMesh::Parallel::Communicator::rank(), libMesh::Parallel::Communicator::send_receive(), and libMesh::Parallel::Communicator::size().

182 {
183  // This function must be run on all processors at once
184  libmesh_parallel_only(comm);
185 
186  // We need a valid location_map
187 #ifdef DEBUG
188  bool need_map_update = (range_begin != range_end && location_map.empty());
189  comm.max(need_map_update);
190  libmesh_assert(!need_map_update);
191 #endif
192 
193  // Count the objects to ask each processor about
194  std::vector<dof_id_type>
195  ghost_objects_from_proc(comm.size(), 0);
196 
197  for (Iterator it = range_begin; it != range_end; ++it)
198  {
199  DofObjType * obj = *it;
200  libmesh_assert (obj);
201  processor_id_type obj_procid = obj->processor_id();
202  if (obj_procid != DofObject::invalid_processor_id)
203  ghost_objects_from_proc[obj_procid]++;
204  }
205 
206  // Request sets to send to each processor
207  std::vector<std::vector<Real>>
208  requested_objs_x(comm.size()),
209  requested_objs_y(comm.size()),
210  requested_objs_z(comm.size());
211  // Corresponding ids to keep track of
212  std::vector<std::vector<dof_id_type>>
213  requested_objs_id(comm.size());
214 
215  // We know how many objects live on each processor, so reserve()
216  // space for each.
217  for (processor_id_type p=0; p != comm.size(); ++p)
218  if (p != comm.rank())
219  {
220  requested_objs_x[p].reserve(ghost_objects_from_proc[p]);
221  requested_objs_y[p].reserve(ghost_objects_from_proc[p]);
222  requested_objs_z[p].reserve(ghost_objects_from_proc[p]);
223  requested_objs_id[p].reserve(ghost_objects_from_proc[p]);
224  }
225  for (Iterator it = range_begin; it != range_end; ++it)
226  {
227  DofObjType * obj = *it;
228  processor_id_type obj_procid = obj->processor_id();
229  if (obj_procid == comm.rank() ||
230  obj_procid == DofObject::invalid_processor_id)
231  continue;
232 
233  Point p = location_map.point_of(*obj);
234  requested_objs_x[obj_procid].push_back(p(0));
235  requested_objs_y[obj_procid].push_back(p(1));
236  requested_objs_z[obj_procid].push_back(p(2));
237  requested_objs_id[obj_procid].push_back(obj->id());
238  }
239 
240  // Trade requests with other processors
241  for (processor_id_type p=1; p != comm.size(); ++p)
242  {
243  // Trade my requests with processor procup and procdown
244  const processor_id_type procup =
245  cast_int<processor_id_type>
246  ((comm.rank() + p) % comm.size());
247  const processor_id_type procdown =
248  cast_int<processor_id_type>
249  ((comm.size() + comm.rank() - p) %
250  comm.size());
251  std::vector<Real> request_to_fill_x,
252  request_to_fill_y,
253  request_to_fill_z;
254  comm.send_receive(procup, requested_objs_x[procup],
255  procdown, request_to_fill_x);
256  comm.send_receive(procup, requested_objs_y[procup],
257  procdown, request_to_fill_y);
258  comm.send_receive(procup, requested_objs_z[procup],
259  procdown, request_to_fill_z);
260 
261  // Find the local id of each requested object
262  std::vector<dof_id_type> request_to_fill_id(request_to_fill_x.size());
263  for (std::size_t i=0; i != request_to_fill_x.size(); ++i)
264  {
265  Point pt(request_to_fill_x[i],
266  request_to_fill_y[i],
267  request_to_fill_z[i]);
268 
269  // Look for this object in the multimap
270  DofObjType * obj = location_map.find(pt);
271 
272  // We'd better find every object we're asked for
273  libmesh_assert (obj);
274 
275  // Return the object's correct processor id,
276  // and our (correct if it's local) id for it.
277  request_to_fill_id[i] = obj->id();
278  }
279 
280  // Gather whatever data the user wants
281  std::vector<typename SyncFunctor::datum> data;
282  sync.gather_data(request_to_fill_id, data);
283 
284  // Trade back the results
285  std::vector<typename SyncFunctor::datum> received_data;
286  comm.send_receive(procdown, data,
287  procup, received_data);
288  libmesh_assert_equal_to (requested_objs_x[procup].size(),
289  received_data.size());
290 
291  // Let the user process the results
292  sync.act_on_data(requested_objs_id[procup], received_data);
293  }
294 }
uint8_t processor_id_type
Definition: id_types.h:99
libmesh_assert(j)
IterBase * data
Ideally this private member data should have protected access.
template<typename Iterator , typename SyncFunctor >
void libMesh::Parallel::sync_element_data_by_parent_id ( MeshBase mesh,
const Iterator &  range_begin,
const Iterator &  range_end,
SyncFunctor &  sync 
)

Request data about a range of ghost elements uniquely identified by their parent id and which child they are.

Fulfill requests with sync.gather_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data), by resizing and setting the values of the data vector. Respond to fulfillment with sync.act_on_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data) The user must define Parallel::StandardType<sync::datum> if sync::datum isn't a built-in type.

Definition at line 402 of file parallel_ghost_sync.h.

References libMesh::Elem::active(), libMesh::Elem::child_ptr(), libMesh::ParallelObject::comm(), data, libMesh::MeshBase::elem_ref(), libMesh::Elem::has_children(), libMesh::DofObject::id(), libMesh::DofObject::invalid_processor_id, libMesh::libmesh_assert(), libMesh::Elem::parent(), libMesh::DofObject::processor_id(), and libMesh::Elem::which_child_am_i().

Referenced by libMesh::MeshCommunication::make_elems_parallel_consistent().

406 {
407  const Communicator & comm (mesh.comm());
408 
409  // This function must be run on all processors at once
410  libmesh_parallel_only(comm);
411 
412  // Count the objects to ask each processor about
413  std::vector<dof_id_type>
414  ghost_objects_from_proc(comm.size(), 0);
415 
416  for (Iterator it = range_begin; it != range_end; ++it)
417  {
418  DofObject * obj = *it;
419  libmesh_assert (obj);
420  processor_id_type obj_procid = obj->processor_id();
421  if (obj_procid != DofObject::invalid_processor_id)
422  ghost_objects_from_proc[obj_procid]++;
423  }
424 
425  // Request sets to send to each processor
426  std::vector<std::vector<dof_id_type>>
427  requested_objs_id(comm.size()),
428  requested_objs_parent_id(comm.size());
429  std::vector<std::vector<unsigned char>>
430  requested_objs_child_num(comm.size());
431 
432  // We know how many objects live on each processor, so reserve()
433  // space for each.
434  for (processor_id_type p=0; p != comm.size(); ++p)
435  if (p != comm.rank())
436  {
437  requested_objs_id[p].reserve(ghost_objects_from_proc[p]);
438  requested_objs_parent_id[p].reserve(ghost_objects_from_proc[p]);
439  requested_objs_child_num[p].reserve(ghost_objects_from_proc[p]);
440  }
441 
442  for (Iterator it = range_begin; it != range_end; ++it)
443  {
444  Elem * elem = *it;
445  processor_id_type obj_procid = elem->processor_id();
446  if (obj_procid == comm.rank() ||
447  obj_procid == DofObject::invalid_processor_id)
448  continue;
449  const Elem * parent = elem->parent();
450  if (!parent || !elem->active())
451  continue;
452 
453  requested_objs_id[obj_procid].push_back(elem->id());
454  requested_objs_parent_id[obj_procid].push_back(parent->id());
455  requested_objs_child_num[obj_procid].push_back
456  (cast_int<unsigned char>
457  (parent->which_child_am_i(elem)));
458  }
459 
460  // Trade requests with other processors
461  for (processor_id_type p=1; p != comm.size(); ++p)
462  {
463  // Trade my requests with processor procup and procdown
464  const processor_id_type procup =
465  cast_int<processor_id_type>
466  ((comm.rank() + p) % comm.size());
467  const processor_id_type procdown =
468  cast_int<processor_id_type>
469  ((comm.size() + comm.rank() - p) %
470  comm.size());
471  std::vector<dof_id_type> request_to_fill_parent_id;
472  std::vector<unsigned char> request_to_fill_child_num;
473  comm.send_receive(procup, requested_objs_parent_id[procup],
474  procdown, request_to_fill_parent_id);
475  comm.send_receive(procup, requested_objs_child_num[procup],
476  procdown, request_to_fill_child_num);
477 
478  // Find the id of each requested element
479  std::size_t request_size = request_to_fill_parent_id.size();
480  std::vector<dof_id_type> request_to_fill_id(request_size);
481  for (std::size_t i=0; i != request_size; ++i)
482  {
483  Elem & parent = mesh.elem_ref(request_to_fill_parent_id[i]);
484  libmesh_assert(parent.has_children());
485  Elem * child = parent.child_ptr(request_to_fill_child_num[i]);
486  libmesh_assert(child);
487  libmesh_assert(child->active());
488  request_to_fill_id[i] = child->id();
489  }
490 
491  // Gather whatever data the user wants
492  std::vector<typename SyncFunctor::datum> data;
493  sync.gather_data(request_to_fill_id, data);
494 
495  // Trade back the results
496  std::vector<typename SyncFunctor::datum> received_data;
497  comm.send_receive(procdown, data,
498  procup, received_data);
499  libmesh_assert_equal_to (requested_objs_id[procup].size(),
500  received_data.size());
501 
502  // Let the user process the results
503  sync.act_on_data(requested_objs_id[procup], received_data);
504  }
505 }
MeshBase & mesh
uint8_t processor_id_type
Definition: id_types.h:99
libmesh_assert(j)
IterBase * data
Ideally this private member data should have protected access.
template<typename ElemCheckFunctor , typename NodeCheckFunctor , typename SyncFunctor >
void libMesh::Parallel::sync_node_data_by_element_id ( MeshBase mesh,
const MeshBase::const_element_iterator range_begin,
const MeshBase::const_element_iterator range_end,
const ElemCheckFunctor &  elem_check,
const NodeCheckFunctor &  node_check,
SyncFunctor &  sync 
)

Request data about a range of ghost nodes uniquely identified by an element id and local node id.

Data for all nodes connected to elements in the given range of element iterators will be requested.

Elements can be further excluded from the request by returning false from element_check(elem)

Nodes can be further excluded from the request by returning false from node_check(elem, local_node_num)

Fulfill requests with sync.gather_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data), by resizing and setting the values of the data vector. Respond to fulfillment with bool sync.act_on_data(const std::vector<unsigned int> & ids, std::vector<sync::datum> & data) and return true iff the response changed any data. The user must define Parallel::StandardType<sync::datum> if sync::datum isn't a built-in type.

Definition at line 522 of file parallel_ghost_sync.h.

References libMesh::ParallelObject::comm(), data, libMesh::MeshBase::elem_ref(), libMesh::DofObject::id(), libMesh::DofObject::invalid_processor_id, libMesh::libmesh_assert(), libMesh::Elem::n_nodes(), libMesh::Elem::node_index_range(), libMesh::Elem::node_ref(), and libMesh::DofObject::processor_id().

Referenced by libMesh::MeshCommunication::make_new_node_proc_ids_parallel_consistent(), libMesh::MeshCommunication::make_node_ids_parallel_consistent(), and libMesh::MeshCommunication::make_node_proc_ids_parallel_consistent().

528 {
529  const Communicator & comm (mesh.comm());
530 
531  // This function must be run on all processors at once
532  libmesh_parallel_only(comm);
533 
534  // Keep track of which nodes we've asked about, so we only hit each
535  // once?
536  // std::unordered_set<dof_id_type> queried_nodes;
537 
538  // No. We need to ask every neighboring processor about every node,
539  // probably repeatedly. Imagine a vertex surrounded by triangles,
540  // each on a different processor, with a ghosting policy that
541  // include only face neighbors and not point neighbors. Then the
542  // only way for authoritative information to trickle out from that
543  // vertex is by being passed along, one neighbor at a time, to
544  // processors who mostly don't even see the node's true owner!
545 
546  bool need_sync = false;
547 
548  do
549  {
550  // This is the last sync we need, unless we later discover
551  // otherwise
552  need_sync = false;
553 
554  // Count the objects to ask each processor about
555  std::vector<dof_id_type>
556  ghost_objects_from_proc(comm.size(), 0);
557 
558  for (MeshBase::const_element_iterator it = range_begin;
559  it != range_end; ++it)
560  {
561  const Elem * elem = *it;
562  libmesh_assert (elem);
563 
564  if (!elem_check(elem))
565  continue;
566 
567  const processor_id_type proc_id = elem->processor_id();
568  if (proc_id == comm.rank() ||
569  proc_id == DofObject::invalid_processor_id)
570  continue;
571 
572  for (auto n : elem->node_index_range())
573  {
574  if (!node_check(elem, n))
575  continue;
576 
577  ghost_objects_from_proc[proc_id]++;
578  }
579  }
580 
581  // Now repeat that iteration, filling request sets this time.
582 
583  // Request sets to send to each processor
584  std::vector<std::vector<dof_id_type>>
585  requested_objs_elem_id(comm.size());
586  std::vector<std::vector<unsigned char>>
587  requested_objs_node_num(comm.size());
588 
589  // Keep track of current local ids for each too
590  std::vector<std::vector<dof_id_type>>
591  requested_objs_id(comm.size());
592 
593  // We know how many objects live on each processor, so reserve()
594  // space for each.
595  for (processor_id_type p=0; p != comm.size(); ++p)
596  if (p != comm.rank())
597  {
598  requested_objs_elem_id[p].reserve(ghost_objects_from_proc[p]);
599  requested_objs_node_num[p].reserve(ghost_objects_from_proc[p]);
600  requested_objs_id[p].reserve(ghost_objects_from_proc[p]);
601  }
602 
603  for (MeshBase::const_element_iterator it = range_begin;
604  it != range_end; ++it)
605  {
606  const Elem * elem = *it;
607  libmesh_assert (elem);
608 
609  if (!elem_check(elem))
610  continue;
611 
612  const processor_id_type proc_id = elem->processor_id();
613  if (proc_id == comm.rank() ||
614  proc_id == DofObject::invalid_processor_id)
615  continue;
616 
617  const dof_id_type elem_id = elem->id();
618 
619  for (auto n : elem->node_index_range())
620  {
621  if (!node_check(elem, n))
622  continue;
623 
624  const Node & node = elem->node_ref(n);
625  const dof_id_type node_id = node.id();
626 
627  requested_objs_elem_id[proc_id].push_back(elem_id);
628  requested_objs_node_num[proc_id].push_back
629  (cast_int<unsigned char>(n));
630  requested_objs_id[proc_id].push_back(node_id);
631  }
632  }
633 
634  // Trade requests with other processors
635  for (processor_id_type p=1; p != comm.size(); ++p)
636  {
637  // Trade my requests with processor procup and procdown
638  const processor_id_type procup =
639  cast_int<processor_id_type>
640  ((comm.rank() + p) % comm.size());
641  const processor_id_type procdown =
642  cast_int<processor_id_type>
643  ((comm.size() + comm.rank() - p) %
644  comm.size());
645 
646  libmesh_assert_equal_to (requested_objs_id[procup].size(),
647  ghost_objects_from_proc[procup]);
648  libmesh_assert_equal_to (requested_objs_elem_id[procup].size(),
649  ghost_objects_from_proc[procup]);
650  libmesh_assert_equal_to (requested_objs_node_num[procup].size(),
651  ghost_objects_from_proc[procup]);
652 
653  std::vector<dof_id_type> request_to_fill_elem_id;
654  std::vector<unsigned char> request_to_fill_node_num;
655  comm.send_receive(procup, requested_objs_elem_id[procup],
656  procdown, request_to_fill_elem_id);
657  comm.send_receive(procup, requested_objs_node_num[procup],
658  procdown, request_to_fill_node_num);
659 
660  // Find the id of each requested element
661  std::size_t request_size = request_to_fill_elem_id.size();
662  std::vector<dof_id_type> request_to_fill_id(request_size);
663  for (std::size_t i=0; i != request_size; ++i)
664  {
665  const Elem & elem = mesh.elem_ref(request_to_fill_elem_id[i]);
666 
667  const unsigned int n = request_to_fill_node_num[i];
668  libmesh_assert_less (n, elem.n_nodes());
669 
670  const Node & node = elem.node_ref(n);
671 
672  // This isn't a safe assertion in the case where we're
673  // syncing processor ids
674  // libmesh_assert_equal_to (node->processor_id(), comm.rank());
675 
676  request_to_fill_id[i] = node.id();
677  }
678 
679  // Gather whatever data the user wants
680  std::vector<typename SyncFunctor::datum> data;
681  sync.gather_data(request_to_fill_id, data);
682 
683  // Trade back the results
684  std::vector<typename SyncFunctor::datum> received_data;
685  comm.send_receive(procdown, data,
686  procup, received_data);
687  libmesh_assert_equal_to (requested_objs_elem_id[procup].size(),
688  received_data.size());
689 
690  // Let the user process the results. If any of the results
691  // were different than what the user expected, then we'll
692  // need to sync again just in case this processor has to
693  // pass on the changes to yet another processor.
694  bool data_changed =
695  sync.act_on_data(requested_objs_id[procup], received_data);
696 
697  if (data_changed)
698  need_sync = true;
699  }
700  comm.max(need_sync);
701  } while (need_sync);
702 }
MeshBase & mesh
uint8_t processor_id_type
Definition: id_types.h:99
libmesh_assert(j)
IterBase * data
Ideally this private member data should have protected access.
uint8_t dof_id_type
Definition: id_types.h:64
template<typename Context , typename buffertype , typename OutputIter , typename T >
void libMesh::Parallel::unpack_range ( const std::vector< buffertype > &  buffer,
Context *  context,
OutputIter  out_iter,
const T *   
)

Helper function for range unpacking.

Definition at line 595 of file parallel_implementation.h.

References libMesh::libmesh_assert(), libMesh::Parallel::Packing< T >::packed_size(), and libMesh::Parallel::Packing< T >::unpack().

599 {
600  // Loop through the buffer and unpack each object, returning the
601  // object pointer via the output iterator
602  typename std::vector<buffertype>::const_iterator
603  next_object_start = buffer.begin();
604 
605  while (next_object_start < buffer.end())
606  {
607  *out_iter++ = Parallel::Packing<T>::unpack(next_object_start, context);
608  next_object_start +=
609  Parallel::Packing<T>::packed_size(next_object_start);
610  }
611 
612  // We should have used up the exact amount of data in the buffer
613  libmesh_assert (next_object_start == buffer.end());
614 }
libmesh_assert(j)
template<typename Context , typename buffertype , typename OutputIter , typename T >
void libMesh::Parallel::unpack_range ( const typename std::vector< buffertype > &  buffer,
Context *  context,
OutputIter  out,
const T *  output_type 
)
template<typename T >
bool libMesh::Parallel::verify ( const T &  r,
const Communicator comm = Communicator_World 
)
Status libMesh::Parallel::wait ( Request r)
void libMesh::Parallel::wait ( std::vector< Request > &  r)

Wait for a non-blocking send or receive to finish.

Definition at line 570 of file parallel.h.

References wait().

571 { for (std::size_t i=0; i<r.size(); i++) r[i].wait(); }
void wait(std::vector< Request > &r)
Wait for a non-blocking send or receive to finish.
Definition: parallel.h:570

Variable Documentation

const unsigned int libMesh::Parallel::any_source
const MessageTag libMesh::Parallel::any_tag = MessageTag(MPI_ANY_TAG)

Default message tag ids.

Definition at line 277 of file parallel.h.

Referenced by libMesh::Parallel::Communicator::send_receive().

Parallel::Communicator & libMesh::Parallel::Communicator_World = CommWorld
const MessageTag libMesh::Parallel::no_tag = MessageTag(0)

Definition at line 282 of file parallel.h.

Referenced by libMesh::Parallel::Communicator::send_receive().